2024-09-21T00:57:14.441 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T00:57:14.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T00:57:14.559 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913359' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJd+QEsAN6F0QeGLWdf4ct1cIzmpFyoX+euLcNJ8X088kKP92XBfnJ7OveZsNmrW6YSOdxAp4HWdC0A8I0EcYxI= smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDVGgE1oxepLsf0ucaUlf3WTKtTPKgPa6XaRkDfWg1yOhk/eQxBfN1poHT+GjHNCH04YBo6+suWTDWi77bFgd4= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBChSJ8kTsE4ocGJDjGPNPB5fyYo1rkS2O3zfHCAt1FUslI5JdFcOHEzlYz1pNgHNpxPkWHJk4TcrZBHO2UStta8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T00:57:14.559 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T00:57:14.559 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T00:57:14.560 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T00:57:14.561 INFO:teuthology.task.internal:Checking packages... 2024-09-21T00:57:14.582 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T00:57:14.582 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T00:57:14.582 INFO:teuthology.packaging:ref: None 2024-09-21T00:57:14.582 INFO:teuthology.packaging:tag: None 2024-09-21T00:57:14.582 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T00:57:14.582 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T00:57:14.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T00:57:14.727 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T00:57:14.729 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T00:57:14.742 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T00:57:14.743 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T00:57:14.976 INFO:teuthology.task.internal:Saving configuration 2024-09-21T00:57:14.999 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T00:57:15.005 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T00:57:15.027 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 00:51:08.298855', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJd+QEsAN6F0QeGLWdf4ct1cIzmpFyoX+euLcNJ8X088kKP92XBfnJ7OveZsNmrW6YSOdxAp4HWdC0A8I0EcYxI='} 2024-09-21T00:57:15.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 00:51:08.302044', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHDVGgE1oxepLsf0ucaUlf3WTKtTPKgPa6XaRkDfWg1yOhk/eQxBfN1poHT+GjHNCH04YBo6+suWTDWi77bFgd4='} 2024-09-21T00:57:15.064 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 00:51:08.300875', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBChSJ8kTsE4ocGJDjGPNPB5fyYo1rkS2O3zfHCAt1FUslI5JdFcOHEzlYz1pNgHNpxPkWHJk4TcrZBHO2UStta8='} 2024-09-21T00:57:15.064 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T00:57:15.069 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T00:57:15.069 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T00:57:15.069 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T00:57:15.069 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T00:57:15.153 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa4f8c4c160>, signals=[15]) 2024-09-21T00:57:15.154 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T00:57:15.160 INFO:teuthology.task.internal:Opening connections... 2024-09-21T00:57:15.160 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T00:57:15.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:57:15.236 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T00:57:15.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:57:15.310 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-09-21T00:57:15.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:57:15.386 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T00:57:15.392 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-09-21T00:57:15.410 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-09-21T00:57:15.411 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-09-21T00:57:15.468 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-09-21T00:57:15.468 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-09-21T00:57:15.468 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-09-21T00:57:15.468 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-09-21T00:57:15.468 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T00:57:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T00:57:15.470 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-09-21T00:57:15.494 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-09-21T00:57:15.512 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-09-21T00:57:15.513 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-09-21T00:57:15.570 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-09-21T00:57:15.570 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T00:57:15.571 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T00:57:15.572 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T00:57:15.572 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-09-21T00:57:15.594 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-09-21T00:57:15.613 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-09-21T00:57:15.613 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-09-21T00:57:15.672 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T00:57:15.673 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T00:57:15.673 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-09-21T00:57:15.694 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T00:57:15.704 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T00:57:15.710 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T00:57:15.710 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-09-21T00:57:15.713 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-09-21T00:57:15.715 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-09-21T00:57:15.733 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T00:57:15.739 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T00:57:15.739 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-09-21T00:57:15.770 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-09-21T00:57:15.773 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-09-21T00:57:16.002 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T00:57:16.081 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T00:57:16.095 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T00:57:16.095 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-21T00:57:16.095 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-21T00:57:16.096 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T00:57:16.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T00:57:16.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T00:57:16.273 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T00:57:16.274 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T00:57:16.274 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-21T00:57:16.293 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-503.el9.x86_64 2024-09-21T00:57:16.293 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T00:57:16.293 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T00:57:16.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2024-09-21T00:57:19.480 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.3 MB 00:01 2024-09-21T00:57:24.938 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 20 MB 00:04 2024-09-21T00:57:30.281 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-21T00:57:31.690 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 131 kB/s | 19 kB 00:00 2024-09-21T00:57:32.947 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T00:57:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T00:57:40.650 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-21T00:57:40.659 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-21T00:57:40.659 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-09-21T00:57:40.659 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-21T00:57:40.659 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 1 Package 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 388 M 2024-09-21T00:57:40.660 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-21T00:57:46.102 INFO:teuthology.orchestra.run.smithi027.stdout:linux-firmware-20240905-146.el9.noarch.rpm 75 MB/s | 388 MB 00:05 2024-09-21T00:57:46.102 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T00:57:46.102 INFO:teuthology.orchestra.run.smithi027.stdout:Total 71 MB/s | 388 MB 00:05 2024-09-21T00:57:49.373 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-21T00:57:49.464 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-21T00:57:49.464 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-21T00:57:52.897 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-21T00:57:52.898 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-21T00:58:02.910 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-21T00:58:03.269 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T00:58:03.362 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T00:58:04.111 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T00:58:04.111 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T00:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T00:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-21T00:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T00:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-21T00:58:05.157 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T00:58:05.157 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T00:58:05.306 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T00:58:05.306 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T00:58:05.309 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2024-09-21T00:58:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21 00:58:11 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T00:58:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2024-09-21 00:58:11-- 2024-09-21T00:58:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 6.4s 2024-09-21T00:58:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 343M in 6.1s (56.2 MB/s) 2024-09-21T00:58:11.716 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T00:58:11.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T00:58:13.885 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2024-09-21T00:58:13.942 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2024-09-21T00:58:14.006 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2024-09-21T00:58:22.354 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T00:59:03.502 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T00:59:03.503 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T00:59:03.576 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.576 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T00:59:03.577 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T00:59:03.578 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T00:59:03.579 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T00:59:03.580 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T00:59:03.581 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T00:59:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T00:59:03.583 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T00:59:03.584 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T00:59:03.585 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T00:59:03.586 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T00:59:03.587 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T00:59:03.588 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T00:59:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T00:59:03.590 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T00:59:03.591 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T00:59:03.592 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T00:59:03.593 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T00:59:03.594 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T00:59:03.595 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T00:59:03.596 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T00:59:03.597 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T00:59:03.598 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T00:59:03.599 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T00:59:03.600 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T00:59:03.601 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T00:59:03.602 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T00:59:03.603 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T00:59:03.604 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T00:59:03.605 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T00:59:03.606 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T00:59:03.607 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T00:59:03.608 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T00:59:03.609 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T00:59:03.610 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T00:59:03.611 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T00:59:03.612 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T00:59:03.613 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T00:59:03.614 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T00:59:03.615 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T00:59:03.616 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T00:59:03.617 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T00:59:03.618 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T00:59:03.619 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T00:59:03.620 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T00:59:03.621 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T00:59:03.622 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T00:59:03.623 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T00:59:03.624 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T00:59:03.625 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T00:59:03.626 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T00:59:03.627 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T00:59:03.628 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T00:59:03.629 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T00:59:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T00:59:03.631 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T00:59:03.632 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T00:59:03.633 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T00:59:03.634 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T00:59:03.635 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T00:59:03.636 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T00:59:03.637 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T00:59:03.638 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T00:59:03.639 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T00:59:03.640 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T00:59:03.641 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T00:59:03.642 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T00:59:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T00:59:03.644 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T00:59:03.645 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T00:59:03.646 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T00:59:03.647 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T00:59:03.648 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T00:59:03.649 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T00:59:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T00:59:03.651 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T00:59:03.652 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T00:59:03.653 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T00:59:03.654 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T00:59:03.655 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T00:59:03.656 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T00:59:03.657 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T00:59:03.658 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T00:59:03.659 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T00:59:03.660 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T00:59:03.661 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T00:59:03.662 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T00:59:03.663 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T00:59:03.664 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T00:59:03.665 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T00:59:03.666 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T00:59:03.667 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T00:59:03.668 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T00:59:03.669 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T00:59:03.670 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T00:59:03.671 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T00:59:03.672 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T00:59:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T00:59:03.674 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T00:59:03.675 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T00:59:03.676 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T00:59:03.677 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T00:59:03.678 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T00:59:03.679 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T00:59:03.680 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T00:59:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T00:59:03.682 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T00:59:03.683 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T00:59:03.684 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T00:59:03.685 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T00:59:03.686 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T00:59:03.687 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T00:59:03.688 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T00:59:03.689 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T00:59:03.690 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T00:59:03.691 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T00:59:03.692 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T00:59:03.693 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T00:59:03.694 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T00:59:03.695 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T00:59:03.696 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T00:59:03.697 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T00:59:03.698 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T00:59:03.699 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T00:59:03.700 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T00:59:03.701 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T00:59:03.702 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T00:59:03.703 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T00:59:03.704 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T00:59:03.705 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T00:59:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T00:59:03.710 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.710 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T00:59:03.734 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T00:59:03.734 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:for file in vmlinuz System.map config; do 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: fi 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:done 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T00:59:03.735 INFO:teuthology.orchestra.run.smithi027.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.736 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T00:59:03.736 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T00:59:03.736 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T00:59:03.736 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.736 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2024-09-21T00:59:03.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-09-21T00:59:03.819 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-09-21T00:59:03.819 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 82.el9 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8274670 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T00:59:03.820 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T00:59:03.821 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T00:59:03.822 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T00:59:03.822 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:03.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T00:59:04.417 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-09-21T00:59:05.888 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T00:59:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-09-21T00:59:05.926 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-09-21T00:59:05.941 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.YLOVFt3SEZ 2024-09-21T00:59:05.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YLOVFt3SEZ 2024-09-21T00:59:06.009 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.YLOVFt3SEZ 2024-09-21T00:59:06.149 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.YLOVFt3SEZ is 6KB 2024-09-21T00:59:06.197 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.YLOVFt3SEZ 2024-09-21T00:59:06.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T00:59:06.277 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T00:59:06.277 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T00:59:06.278 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T00:59:06.278 INFO:teuthology.orchestra.run.smithi027.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T00:59:06.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T00:59:06.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-09-21T00:59:36.438 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T00:59:36.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T00:59:36.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T00:59:54.991 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T01:00:03.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:00:03.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:00:07.055 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T01:00:19.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:00:19.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:00:22.127 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T01:00:37.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:00:37.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:00:40.207 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T01:00:58.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:00:58.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:01:16.655 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-09-21T01:01:37.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:01:37.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:01:37.937 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-21T01:01:38.410 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:01:38.410 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T01:01:38.411 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-09-21T01:01:38.444 INFO:teuthology.orchestra.run.smithi027.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T01:01:38.445 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T01:01:38.445 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T01:01:38.445 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T01:01:39.445 DEBUG:teuthology.parallel:result is None 2024-09-21T01:01:39.445 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T01:01:39.457 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T01:01:39.457 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:01:39.470 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:01:39.473 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T01:01:39.536 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T01:01:39.542 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T01:01:39.550 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T01:01:39.550 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:01:39.559 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:01:39.561 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T01:01:39.679 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T01:01:39.685 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T01:01:39.685 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:01:39.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:01:39.702 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T01:01:39.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:01:39.719 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:01:39.728 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:01:39.745 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T01:01:39.783 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:01:39.802 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:01:39.803 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:01:39.818 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:01:39.933 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:01:40.005 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T01:01:40.009 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T01:01:40.016 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T01:01:40.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:01:40.025 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:01:40.027 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T01:01:40.138 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T01:01:40.147 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T01:01:40.147 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:01:40.155 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:01:40.158 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T01:01:40.219 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:01:40.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:01:40.462 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:01:40.564 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:01:40.688 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:01:40.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:01:40.800 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:01:40.839 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:01:40.902 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:01:40.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:01:41.031 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:01:41.031 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:01:41.098 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:01:41.140 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T01:01:41.427 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:01:41.469 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T01:01:41.517 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:01:41.518 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T01:01:41.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-09-21T01:01:41.589 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-09-21T01:01:41.592 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-09-21T01:01:41.637 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:01:41.665 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:01:41.681 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:01:42.040 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T01:01:42.048 INFO:teuthology.task.internal:Starting timer... 2024-09-21T01:01:42.048 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T01:01:42.057 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T01:01:42.066 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T01:01:42.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-09-21T01:01:42.162 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-09-21T01:01:42.166 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-09-21T01:01:42.217 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-09-21T01:01:42.219 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-09-21T01:01:42.276 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-09-21T01:01:42.278 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T01:01:42.278 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-09-21T01:01:42.319 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-09-21T01:01:42.320 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-09-21T01:01:42.345 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-09-21T01:01:42.345 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-09-21T01:01:42.373 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-09-21T01:01:42.374 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive'} 2024-09-21T01:01:42.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:01:42.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:01:42.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:01:42.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:01:42.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:01:42.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:01:42.492 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T01:01:42.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:01:42.553 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:01:42.593 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-09-21T01:01:42.623 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T01:01:42.632 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T01:01:42.635 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T01:01:42.774 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T01:01:42.784 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T01:01:42.785 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi089.front.sepia.ceph.com,smithi098.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T01:06:26.990 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com')] 2024-09-21T01:06:26.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:06:26.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:06:27.081 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-09-21T01:06:27.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-09-21T01:06:27.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T01:06:27.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:06:27.257 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-09-21T01:06:27.342 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-09-21T01:06:27.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T01:06:27.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T01:06:27.421 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-21T01:06:27.504 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-21T01:06:27.505 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T01:06:27.515 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T01:06:27.515 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:06:27.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:06:27.519 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:06:27.519 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:06:27.521 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T01:06:27.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:06:27.549 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:06:27.564 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:06:27.577 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:06:27.578 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T01:06:27.588 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-09-21T01:06:27.592 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:06:27.598 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-09-21T01:06:27.608 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T01:06:27.613 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:06:27.618 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-09-21T01:06:27.626 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:06:27.629 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-09-21T01:06:27.644 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:06:27.658 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:06:27.660 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-09-21T01:06:27.684 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-09-21T01:06:27.688 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:06:27.691 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:06:27.691 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-21T01:06:27.691 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.691 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.691 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.691 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.713 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:06:27.714 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T01:06:27.717 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:06:27.717 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-21T01:06:27.717 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.717 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.717 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.717 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.744 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T01:06:27.836 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:06:27.843 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:06:27.843 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-21T01:06:27.843 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.843 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.844 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.844 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:06:27.844 INFO:teuthology.run_tasks:Running task install... 2024-09-21T01:06:27.853 DEBUG:teuthology.task.install:project ceph 2024-09-21T01:06:27.853 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T01:06:27.853 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T01:06:27.853 INFO:teuthology.task.install:Using flavor: default 2024-09-21T01:06:27.858 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T01:06:27.858 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T01:06:27.858 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:06:27.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:06:27.860 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:06:27.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:06:27.861 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T01:06:27.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:06:28.016 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:06:28.016 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:06:28.028 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:06:28.028 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:06:28.034 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T01:06:28.034 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T01:06:28.126 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:06:28.126 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:06:28.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:06:28.136 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:06:28.136 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:06:28.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:06:28.146 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T01:06:28.146 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:06:28.146 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T01:06:28.163 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:06:28.163 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:06:28.172 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:06:28.172 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:06:28.185 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T01:06:28.185 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T01:06:28.228 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:06:28.239 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:06:28.283 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T01:06:28.304 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:06:28.313 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:06:28.364 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T01:06:28.370 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-09-21T01:06:28.372 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-09-21T01:06:28.377 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-09-21T01:06:28.378 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-09-21T01:06:28.420 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-09-21T01:06:28.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-09-21T01:06:28.598 INFO:teuthology.orchestra.run.smithi098.stdout:45 files removed 2024-09-21T01:06:28.602 INFO:teuthology.orchestra.run.smithi089.stdout:45 files removed 2024-09-21T01:06:28.625 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:06:28.629 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:06:28.699 INFO:teuthology.orchestra.run.smithi027.stdout:45 files removed 2024-09-21T01:06:28.733 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T01:06:29.175 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-21T01:06:29.178 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T01:06:29.443 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 288 kB/s | 84 kB 00:00 2024-09-21T01:06:29.462 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T01:06:29.467 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T01:06:29.737 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T01:06:29.743 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T01:06:29.777 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 84 kB/s | 24 kB 00:00 2024-09-21T01:06:30.061 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T01:06:30.221 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-21T01:06:31.668 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-21T01:06:32.173 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-21T01:06:32.994 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.3 MB 00:02 2024-09-21T01:06:34.762 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-21T01:06:35.870 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-21T01:06:37.490 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-21T01:06:39.133 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-21T01:06:40.270 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T01:06:41.087 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 12 MB/s | 23 MB 00:01 2024-09-21T01:06:41.546 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-21T01:06:41.971 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-21T01:06:43.180 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-21T01:06:43.211 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T01:06:44.068 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T01:06:46.602 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T01:06:48.748 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:06:48.749 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:06:48.762 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:06:48.763 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:06:48.764 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:06:48.765 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:06:48.767 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:06:48.770 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:06:48.771 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:06:48.806 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T01:06:48.834 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-09-21T01:06:48.851 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T01:06:48.851 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-09-21T01:06:48.851 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T01:06:48.851 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-09-21T01:06:48.851 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:06:48.851 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:06:48.852 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:06:48.853 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:06:48.854 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:06:48.855 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:06:48.856 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:06:48.857 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:06:48.858 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:06:48.859 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:06:48.860 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:06:48.861 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:06:48.862 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:06:48.863 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:06:48.864 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:06:48.865 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:06:48.866 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:06:48.867 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-09-21T01:06:48.867 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-09-21T01:06:48.867 INFO:teuthology.orchestra.run.smithi089.stdout:Install 149 Packages 2024-09-21T01:06:48.867 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 8 Packages 2024-09-21T01:06:48.867 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:06:48.868 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 228 M 2024-09-21T01:06:48.868 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-09-21T01:06:49.229 INFO:teuthology.orchestra.run.smithi089.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-09-21T01:06:49.404 INFO:teuthology.orchestra.run.smithi089.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.9 MB/s | 873 kB 00:00 2024-09-21T01:06:49.513 INFO:teuthology.orchestra.run.smithi089.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.3 MB/s | 148 kB 00:00 2024-09-21T01:06:49.621 INFO:teuthology.orchestra.run.smithi089.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 10 MB/s | 5.5 MB 00:00 2024-09-21T01:06:49.688 INFO:teuthology.orchestra.run.smithi089.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 13 MB/s | 2.3 MB 00:00 2024-09-21T01:06:49.730 INFO:teuthology.orchestra.run.smithi089.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 14 MB/s | 1.5 MB 00:00 2024-09-21T01:06:50.205 INFO:teuthology.orchestra.run.smithi089.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T01:06:50.281 INFO:teuthology.orchestra.run.smithi089.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 8.4 MB/s | 5.0 MB 00:00 2024-09-21T01:06:50.331 INFO:teuthology.orchestra.run.smithi089.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 505 kB/s | 25 kB 00:00 2024-09-21T01:06:50.657 INFO:teuthology.orchestra.run.smithi089.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 23 MB/s | 10 MB 00:00 2024-09-21T01:06:50.716 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-21T01:06:50.849 INFO:teuthology.orchestra.run.smithi089.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 16 MB/s | 18 MB 00:01 2024-09-21T01:06:50.882 INFO:teuthology.orchestra.run.smithi089.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.0 MB/s | 226 kB 00:00 2024-09-21T01:06:50.899 INFO:teuthology.orchestra.run.smithi089.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 541 kB/s | 27 kB 00:00 2024-09-21T01:06:50.916 INFO:teuthology.orchestra.run.smithi089.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 943 kB/s | 32 kB 00:00 2024-09-21T01:06:50.966 INFO:teuthology.orchestra.run.smithi089.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T01:06:50.992 INFO:teuthology.orchestra.run.smithi089.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T01:06:51.000 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:06:51.001 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:06:51.012 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:06:51.013 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:06:51.014 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:06:51.015 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:06:51.016 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:06:51.017 INFO:teuthology.orchestra.run.smithi089.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T01:06:51.019 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:06:51.020 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:06:51.050 INFO:teuthology.orchestra.run.smithi089.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T01:06:51.083 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-09-21T01:06:51.097 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T01:06:51.097 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-09-21T01:06:51.097 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T01:06:51.097 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:06:51.098 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:06:51.099 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:06:51.100 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:06:51.101 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:06:51.102 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:06:51.103 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:06:51.104 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:06:51.105 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:06:51.106 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:06:51.107 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:06:51.108 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:06:51.109 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout:Install 149 Packages 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 8 Packages 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 228 M 2024-09-21T01:06:51.110 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-09-21T01:06:51.118 INFO:teuthology.orchestra.run.smithi089.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 676 kB/s | 45 kB 00:00 2024-09-21T01:06:51.193 INFO:teuthology.orchestra.run.smithi089.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T01:06:51.277 INFO:teuthology.orchestra.run.smithi089.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.3 MB 00:00 2024-09-21T01:06:51.302 INFO:teuthology.orchestra.run.smithi089.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.5 MB/s | 161 kB 00:00 2024-09-21T01:06:51.336 INFO:teuthology.orchestra.run.smithi089.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T01:06:51.368 INFO:teuthology.orchestra.run.smithi089.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.6 MB/s | 307 kB 00:00 2024-09-21T01:06:51.393 INFO:teuthology.orchestra.run.smithi089.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T01:06:51.419 INFO:teuthology.orchestra.run.smithi089.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T01:06:51.477 INFO:teuthology.orchestra.run.smithi089.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T01:06:51.544 INFO:teuthology.orchestra.run.smithi089.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T01:06:51.570 INFO:teuthology.orchestra.run.smithi089.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 371 kB/s | 34 kB 00:00 2024-09-21T01:06:51.595 INFO:teuthology.orchestra.run.smithi089.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T01:06:52.033 INFO:teuthology.orchestra.run.smithi098.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-09-21T01:06:52.037 INFO:teuthology.orchestra.run.smithi089.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 17 MB/s | 7.4 MB 00:00 2024-09-21T01:06:52.104 INFO:teuthology.orchestra.run.smithi089.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T01:06:52.254 INFO:teuthology.orchestra.run.smithi089.stdout:(33/157): ceph-mgr-dashboard-19.3.0-5120.g18432 21 MB/s | 14 MB 00:00 2024-09-21T01:06:52.280 INFO:teuthology.orchestra.run.smithi089.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 281 kB/s | 49 kB 00:00 2024-09-21T01:06:52.305 INFO:teuthology.orchestra.run.smithi089.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T01:06:52.339 INFO:teuthology.orchestra.run.smithi089.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T01:06:52.359 INFO:teuthology.orchestra.run.smithi098.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.6 MB/s | 873 kB 00:00 2024-09-21T01:06:52.372 INFO:teuthology.orchestra.run.smithi089.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 11 MB/s | 784 kB 00:00 2024-09-21T01:06:52.398 INFO:teuthology.orchestra.run.smithi089.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 314 kB/s | 18 kB 00:00 2024-09-21T01:06:52.425 INFO:teuthology.orchestra.run.smithi098.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T01:06:52.490 INFO:teuthology.orchestra.run.smithi089.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 683 kB/s | 80 kB 00:00 2024-09-21T01:06:52.548 INFO:teuthology.orchestra.run.smithi089.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T01:06:52.582 INFO:teuthology.orchestra.run.smithi089.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-21T01:06:52.607 INFO:teuthology.orchestra.run.smithi089.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-21T01:06:52.649 INFO:teuthology.orchestra.run.smithi089.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T01:06:52.716 INFO:teuthology.orchestra.run.smithi089.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-21T01:06:52.734 INFO:teuthology.orchestra.run.smithi098.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 6.0 MB/s | 5.5 MB 00:00 2024-09-21T01:06:52.775 INFO:teuthology.orchestra.run.smithi089.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T01:06:52.826 INFO:teuthology.orchestra.run.smithi098.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 5.7 MB/s | 2.3 MB 00:00 2024-09-21T01:06:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-09-21T01:06:53.159 INFO:teuthology.orchestra.run.smithi089.stdout:(46/157): ceph-test-19.3.0-5120.g18432218.el9.x 14 MB/s | 40 MB 00:02 2024-09-21T01:06:53.180 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:06:53.180 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T01:06:53.184 INFO:teuthology.orchestra.run.smithi089.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 525 kB/s | 215 kB 00:00 2024-09-21T01:06:53.193 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T01:06:53.194 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T01:06:53.195 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T01:06:53.195 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T01:06:53.197 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T01:06:53.200 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T01:06:53.201 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T01:06:53.218 INFO:teuthology.orchestra.run.smithi089.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-09-21T01:06:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-09-21T01:06:53.276 INFO:teuthology.orchestra.run.smithi098.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-21T01:06:53.292 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T01:06:53.292 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-09-21T01:06:53.292 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T01:06:53.292 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T01:06:53.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T01:06:53.294 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T01:06:53.295 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T01:06:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T01:06:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T01:06:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T01:06:53.296 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-09-21T01:06:53.296 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T01:06:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T01:06:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T01:06:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T01:06:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T01:06:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T01:06:53.298 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T01:06:53.299 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T01:06:53.300 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T01:06:53.301 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T01:06:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T01:06:53.303 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T01:06:53.304 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T01:06:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T01:06:53.306 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout:Install 149 Packages 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 8 Packages 2024-09-21T01:06:53.307 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:06:53.308 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 228 M 2024-09-21T01:06:53.308 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-09-21T01:06:53.326 INFO:teuthology.orchestra.run.smithi089.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-21T01:06:53.377 INFO:teuthology.orchestra.run.smithi089.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-09-21T01:06:53.402 INFO:teuthology.orchestra.run.smithi089.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 581 kB/s | 107 kB 00:00 2024-09-21T01:06:53.427 INFO:teuthology.orchestra.run.smithi089.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-09-21T01:06:53.453 INFO:teuthology.orchestra.run.smithi089.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 313 kB 00:00 2024-09-21T01:06:53.478 INFO:teuthology.orchestra.run.smithi089.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 431 kB/s | 33 kB 00:00 2024-09-21T01:06:53.520 INFO:teuthology.orchestra.run.smithi089.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 271 kB/s | 18 kB 00:00 2024-09-21T01:06:53.545 INFO:teuthology.orchestra.run.smithi089.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-21T01:06:53.596 INFO:teuthology.orchestra.run.smithi089.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-21T01:06:53.621 INFO:teuthology.orchestra.run.smithi089.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2024-09-21T01:06:53.646 INFO:teuthology.orchestra.run.smithi089.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-21T01:06:53.671 INFO:teuthology.orchestra.run.smithi089.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-09-21T01:06:53.705 INFO:teuthology.orchestra.run.smithi089.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T01:06:53.731 INFO:teuthology.orchestra.run.smithi089.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-21T01:06:53.756 INFO:teuthology.orchestra.run.smithi089.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 291 kB/s | 24 kB 00:00 2024-09-21T01:06:53.768 INFO:teuthology.orchestra.run.smithi098.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 10 MB/s | 20 MB 00:01 2024-09-21T01:06:53.781 INFO:teuthology.orchestra.run.smithi089.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-21T01:06:53.806 INFO:teuthology.orchestra.run.smithi089.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T01:06:53.840 INFO:teuthology.orchestra.run.smithi089.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T01:06:53.865 INFO:teuthology.orchestra.run.smithi089.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 457 kB/s | 38 kB 00:00 2024-09-21T01:06:53.957 INFO:teuthology.orchestra.run.smithi089.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-21T01:06:53.961 INFO:teuthology.orchestra.run.smithi098.stdout:(9/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 17 MB/s | 18 MB 00:01 2024-09-21T01:06:53.986 INFO:teuthology.orchestra.run.smithi098.stdout:(10/157): ceph-selinux-19.3.0-5120.g18432218.el 116 kB/s | 25 kB 00:00 2024-09-21T01:06:54.000 INFO:teuthology.orchestra.run.smithi089.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T01:06:54.033 INFO:teuthology.orchestra.run.smithi089.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T01:06:54.100 INFO:teuthology.orchestra.run.smithi089.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-21T01:06:54.119 INFO:teuthology.orchestra.run.smithi098.stdout:(11/157): ceph-radosgw-19.3.0-5120.g18432218.el 12 MB/s | 10 MB 00:00 2024-09-21T01:06:54.125 INFO:teuthology.orchestra.run.smithi089.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T01:06:54.145 INFO:teuthology.orchestra.run.smithi098.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.4 MB/s | 226 kB 00:00 2024-09-21T01:06:54.151 INFO:teuthology.orchestra.run.smithi027.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 41 kB/s | 6.6 kB 00:00 2024-09-21T01:06:54.170 INFO:teuthology.orchestra.run.smithi098.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 541 kB/s | 27 kB 00:00 2024-09-21T01:06:54.195 INFO:teuthology.orchestra.run.smithi098.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 634 kB/s | 32 kB 00:00 2024-09-21T01:06:54.226 INFO:teuthology.orchestra.run.smithi089.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-09-21T01:06:54.237 INFO:teuthology.orchestra.run.smithi098.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T01:06:54.251 INFO:teuthology.orchestra.run.smithi089.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 699 kB/s | 87 kB 00:00 2024-09-21T01:06:54.262 INFO:teuthology.orchestra.run.smithi098.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T01:06:54.285 INFO:teuthology.orchestra.run.smithi089.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-09-21T01:06:54.296 INFO:teuthology.orchestra.run.smithi098.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T01:06:54.327 INFO:teuthology.orchestra.run.smithi089.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 812 kB/s | 61 kB 00:00 2024-09-21T01:06:54.346 INFO:teuthology.orchestra.run.smithi098.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 5.9 MB/s | 507 kB 00:00 2024-09-21T01:06:54.402 INFO:teuthology.orchestra.run.smithi089.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-21T01:06:54.427 INFO:teuthology.orchestra.run.smithi089.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-21T01:06:54.452 INFO:teuthology.orchestra.run.smithi089.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-09-21T01:06:54.486 INFO:teuthology.orchestra.run.smithi089.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 304 kB/s | 26 kB 00:00 2024-09-21T01:06:54.512 INFO:teuthology.orchestra.run.smithi089.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 227 kB/s | 19 kB 00:00 2024-09-21T01:06:54.537 INFO:teuthology.orchestra.run.smithi089.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T01:06:54.560 INFO:teuthology.orchestra.run.smithi027.stdout:(2/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.7 MB/s | 5.5 MB 00:00 2024-09-21T01:06:54.562 INFO:teuthology.orchestra.run.smithi089.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-09-21T01:06:54.621 INFO:teuthology.orchestra.run.smithi089.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-21T01:06:54.635 INFO:teuthology.orchestra.run.smithi027.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.9 MB/s | 148 kB 00:00 2024-09-21T01:06:54.646 INFO:teuthology.orchestra.run.smithi089.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T01:06:54.688 INFO:teuthology.orchestra.run.smithi089.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-21T01:06:54.705 INFO:teuthology.orchestra.run.smithi089.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-21T01:06:54.710 INFO:teuthology.orchestra.run.smithi027.stdout:(4/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 873 kB 00:00 2024-09-21T01:06:54.764 INFO:teuthology.orchestra.run.smithi089.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-21T01:06:54.790 INFO:teuthology.orchestra.run.smithi089.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-21T01:06:54.827 INFO:teuthology.orchestra.run.smithi027.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 12 MB/s | 2.3 MB 00:00 2024-09-21T01:06:54.856 INFO:teuthology.orchestra.run.smithi089.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-21T01:06:54.863 INFO:teuthology.orchestra.run.smithi098.stdout:(19/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.3 MB/s | 5.3 MB 00:00 2024-09-21T01:06:54.880 INFO:teuthology.orchestra.run.smithi098.stdout:(20/157): python3-ceph-argparse-19.3.0-5120.g18 85 kB/s | 45 kB 00:00 2024-09-21T01:06:54.881 INFO:teuthology.orchestra.run.smithi089.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-09-21T01:06:54.915 INFO:teuthology.orchestra.run.smithi089.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-21T01:06:54.940 INFO:teuthology.orchestra.run.smithi089.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 54 kB 00:00 2024-09-21T01:06:54.956 INFO:teuthology.orchestra.run.smithi098.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.6 MB/s | 148 kB 00:00 2024-09-21T01:06:54.974 INFO:teuthology.orchestra.run.smithi089.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-21T01:06:54.984 INFO:teuthology.orchestra.run.smithi098.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.5 MB/s | 161 kB 00:00 2024-09-21T01:06:55.008 INFO:teuthology.orchestra.run.smithi089.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-09-21T01:06:55.041 INFO:teuthology.orchestra.run.smithi089.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T01:06:55.060 INFO:teuthology.orchestra.run.smithi098.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.0 MB/s | 321 kB 00:00 2024-09-21T01:06:55.076 INFO:teuthology.orchestra.run.smithi089.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-09-21T01:06:55.093 INFO:teuthology.orchestra.run.smithi098.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 2.8 MB/s | 307 kB 00:00 2024-09-21T01:06:55.117 INFO:teuthology.orchestra.run.smithi089.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-21T01:06:55.127 INFO:teuthology.orchestra.run.smithi098.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.4 MB/s | 99 kB 00:00 2024-09-21T01:06:55.144 INFO:teuthology.orchestra.run.smithi027.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 16 MB/s | 5.0 MB 00:00 2024-09-21T01:06:55.151 INFO:teuthology.orchestra.run.smithi089.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.9 MB/s | 304 kB 00:00 2024-09-21T01:06:55.152 INFO:teuthology.orchestra.run.smithi098.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T01:06:55.269 INFO:teuthology.orchestra.run.smithi098.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.4 MB/s | 170 kB 00:00 2024-09-21T01:06:55.326 INFO:teuthology.orchestra.run.smithi089.stdout:(100/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-21T01:06:55.352 INFO:teuthology.orchestra.run.smithi089.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 247 kB/s | 58 kB 00:00 2024-09-21T01:06:55.377 INFO:teuthology.orchestra.run.smithi089.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 424 kB/s | 96 kB 00:00 2024-09-21T01:06:55.378 INFO:teuthology.orchestra.run.smithi098.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 317 kB/s | 34 kB 00:00 2024-09-21T01:06:55.402 INFO:teuthology.orchestra.run.smithi089.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-09-21T01:06:55.427 INFO:teuthology.orchestra.run.smithi089.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 593 kB/s | 45 kB 00:00 2024-09-21T01:06:55.427 INFO:teuthology.orchestra.run.smithi027.stdout:(7/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 2.1 MB/s | 1.5 MB 00:00 2024-09-21T01:06:55.445 INFO:teuthology.orchestra.run.smithi098.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.3 MB/s | 154 kB 00:00 2024-09-21T01:06:55.486 INFO:teuthology.orchestra.run.smithi089.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T01:06:55.511 INFO:teuthology.orchestra.run.smithi089.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T01:06:55.512 INFO:teuthology.orchestra.run.smithi098.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 8.1 MB/s | 3.1 MB 00:00 2024-09-21T01:06:55.545 INFO:teuthology.orchestra.run.smithi089.stdout:(107/157): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-21T01:06:55.570 INFO:teuthology.orchestra.run.smithi089.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 330 kB/s | 19 kB 00:00 2024-09-21T01:06:55.604 INFO:teuthology.orchestra.run.smithi089.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T01:06:55.629 INFO:teuthology.orchestra.run.smithi089.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-21T01:06:55.636 INFO:teuthology.orchestra.run.smithi027.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 12 MB/s | 20 MB 00:01 2024-09-21T01:06:55.663 INFO:teuthology.orchestra.run.smithi089.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T01:06:55.705 INFO:teuthology.orchestra.run.smithi089.stdout:(112/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-09-21T01:06:55.720 INFO:teuthology.orchestra.run.smithi027.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 302 kB/s | 25 kB 00:00 2024-09-21T01:06:55.755 INFO:teuthology.orchestra.run.smithi089.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-21T01:06:55.797 INFO:teuthology.orchestra.run.smithi089.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T01:06:55.823 INFO:teuthology.orchestra.run.smithi089.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T01:06:55.848 INFO:teuthology.orchestra.run.smithi089.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T01:06:55.873 INFO:teuthology.orchestra.run.smithi089.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-21T01:06:55.907 INFO:teuthology.orchestra.run.smithi089.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-21T01:06:55.949 INFO:teuthology.orchestra.run.smithi089.stdout:(119/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-09-21T01:06:55.983 INFO:teuthology.orchestra.run.smithi089.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-21T01:06:56.008 INFO:teuthology.orchestra.run.smithi089.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T01:06:56.033 INFO:teuthology.orchestra.run.smithi089.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T01:06:56.059 INFO:teuthology.orchestra.run.smithi089.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T01:06:56.076 INFO:teuthology.orchestra.run.smithi089.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-21T01:06:56.092 INFO:teuthology.orchestra.run.smithi089.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T01:06:56.151 INFO:teuthology.orchestra.run.smithi089.stdout:(126/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-21T01:06:56.177 INFO:teuthology.orchestra.run.smithi089.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-09-21T01:06:56.202 INFO:teuthology.orchestra.run.smithi089.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T01:06:56.227 INFO:teuthology.orchestra.run.smithi089.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T01:06:56.252 INFO:teuthology.orchestra.run.smithi089.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T01:06:56.278 INFO:teuthology.orchestra.run.smithi089.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T01:06:56.303 INFO:teuthology.orchestra.run.smithi089.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T01:06:56.337 INFO:teuthology.orchestra.run.smithi089.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T01:06:56.362 INFO:teuthology.orchestra.run.smithi089.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-21T01:06:56.396 INFO:teuthology.orchestra.run.smithi089.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-21T01:06:56.421 INFO:teuthology.orchestra.run.smithi089.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T01:06:56.447 INFO:teuthology.orchestra.run.smithi089.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-21T01:06:56.472 INFO:teuthology.orchestra.run.smithi089.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T01:06:56.497 INFO:teuthology.orchestra.run.smithi089.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T01:06:56.537 INFO:teuthology.orchestra.run.smithi027.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 13 MB/s | 18 MB 00:01 2024-09-21T01:06:56.537 INFO:teuthology.orchestra.run.smithi098.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 7.2 MB/s | 7.4 MB 00:01 2024-09-21T01:06:56.541 INFO:teuthology.orchestra.run.smithi089.stdout:(140/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-21T01:06:56.565 INFO:teuthology.orchestra.run.smithi089.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-09-21T01:06:56.587 INFO:teuthology.orchestra.run.smithi027.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T01:06:56.590 INFO:teuthology.orchestra.run.smithi089.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-21T01:06:56.615 INFO:teuthology.orchestra.run.smithi089.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-21T01:06:56.642 INFO:teuthology.orchestra.run.smithi089.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T01:06:56.666 INFO:teuthology.orchestra.run.smithi089.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T01:06:56.670 INFO:teuthology.orchestra.run.smithi027.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 325 kB/s | 27 kB 00:00 2024-09-21T01:06:56.691 INFO:teuthology.orchestra.run.smithi089.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-21T01:06:56.717 INFO:teuthology.orchestra.run.smithi089.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T01:06:56.738 INFO:teuthology.orchestra.run.smithi027.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 475 kB/s | 32 kB 00:00 2024-09-21T01:06:56.742 INFO:teuthology.orchestra.run.smithi089.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T01:06:56.784 INFO:teuthology.orchestra.run.smithi089.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T01:06:56.838 INFO:teuthology.orchestra.run.smithi027.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T01:06:56.897 INFO:teuthology.orchestra.run.smithi098.stdout:(32/157): ceph-test-19.3.0-5120.g18432218.el9.x 14 MB/s | 40 MB 00:02 2024-09-21T01:06:56.897 INFO:teuthology.orchestra.run.smithi027.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T01:06:56.921 INFO:teuthology.orchestra.run.smithi098.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 761 kB/s | 292 kB 00:00 2024-09-21T01:06:56.955 INFO:teuthology.orchestra.run.smithi098.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 844 kB/s | 49 kB 00:00 2024-09-21T01:06:56.963 INFO:teuthology.orchestra.run.smithi027.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T01:06:57.063 INFO:teuthology.orchestra.run.smithi027.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 5.0 MB/s | 507 kB 00:00 2024-09-21T01:06:57.097 INFO:teuthology.orchestra.run.smithi098.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 8.6 MB/s | 14 MB 00:01 2024-09-21T01:06:57.101 INFO:teuthology.orchestra.run.smithi089.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.8 MB/s | 3.2 MB 00:00 2024-09-21T01:06:57.122 INFO:teuthology.orchestra.run.smithi098.stdout:(36/157): ceph-prometheus-alerts-19.3.0-5120.g1 82 kB/s | 16 kB 00:00 2024-09-21T01:06:57.147 INFO:teuthology.orchestra.run.smithi098.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 1.4 MB/s | 285 kB 00:00 2024-09-21T01:06:57.160 INFO:teuthology.orchestra.run.smithi089.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 7.7 MB/s | 3.4 MB 00:00 2024-09-21T01:06:57.198 INFO:teuthology.orchestra.run.smithi098.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 245 kB/s | 18 kB 00:00 2024-09-21T01:06:57.231 INFO:teuthology.orchestra.run.smithi098.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 5.7 MB/s | 784 kB 00:00 2024-09-21T01:06:57.252 INFO:teuthology.orchestra.run.smithi089.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 447 kB/s | 41 kB 00:00 2024-09-21T01:06:57.344 INFO:teuthology.orchestra.run.smithi089.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.7 MB/s | 249 kB 00:00 2024-09-21T01:06:57.348 INFO:teuthology.orchestra.run.smithi098.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 398 kB/s | 80 kB 00:00 2024-09-21T01:06:57.364 INFO:teuthology.orchestra.run.smithi027.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.3 MB 00:00 2024-09-21T01:06:57.436 INFO:teuthology.orchestra.run.smithi089.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 3.6 MB/s | 2.3 MB 00:00 2024-09-21T01:06:57.439 INFO:teuthology.orchestra.run.smithi027.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 605 kB/s | 45 kB 00:00 2024-09-21T01:06:57.457 INFO:teuthology.orchestra.run.smithi098.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-09-21T01:06:57.491 INFO:teuthology.orchestra.run.smithi098.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-21T01:06:57.498 INFO:teuthology.orchestra.run.smithi027.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T01:06:57.533 INFO:teuthology.orchestra.run.smithi098.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-21T01:06:57.556 INFO:teuthology.orchestra.run.smithi027.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T01:06:57.559 INFO:teuthology.orchestra.run.smithi098.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T01:06:57.592 INFO:teuthology.orchestra.run.smithi098.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 135 kB 00:00 2024-09-21T01:06:57.617 INFO:teuthology.orchestra.run.smithi098.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T01:06:57.619 INFO:teuthology.orchestra.run.smithi089.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.5 MB/s | 1.8 MB 00:00 2024-09-21T01:06:57.650 INFO:teuthology.orchestra.run.smithi098.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T01:06:57.661 INFO:teuthology.orchestra.run.smithi089.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 1.4 MB 00:00 2024-09-21T01:06:57.717 INFO:teuthology.orchestra.run.smithi098.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-09-21T01:06:57.843 INFO:teuthology.orchestra.run.smithi098.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-21T01:06:57.868 INFO:teuthology.orchestra.run.smithi098.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 711 kB/s | 107 kB 00:00 2024-09-21T01:06:57.893 INFO:teuthology.orchestra.run.smithi098.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-21T01:06:57.935 INFO:teuthology.orchestra.run.smithi098.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-21T01:06:57.960 INFO:teuthology.orchestra.run.smithi098.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-21T01:06:57.973 INFO:teuthology.orchestra.run.smithi027.stdout:(22/157): ceph-test-19.3.0-5120.g18432218.el9.x 18 MB/s | 40 MB 00:02 2024-09-21T01:06:57.999 INFO:teuthology.orchestra.run.smithi027.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 727 kB/s | 321 kB 00:00 2024-09-21T01:06:58.002 INFO:teuthology.orchestra.run.smithi098.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-21T01:06:58.028 INFO:teuthology.orchestra.run.smithi098.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-21T01:06:58.032 INFO:teuthology.orchestra.run.smithi027.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T01:06:58.053 INFO:teuthology.orchestra.run.smithi098.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T01:06:58.057 INFO:teuthology.orchestra.run.smithi027.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T01:06:58.074 INFO:teuthology.orchestra.run.smithi027.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 86 kB 00:00 2024-09-21T01:06:58.078 INFO:teuthology.orchestra.run.smithi098.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T01:06:58.116 INFO:teuthology.orchestra.run.smithi027.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 4.0 MB/s | 170 kB 00:00 2024-09-21T01:06:58.128 INFO:teuthology.orchestra.run.smithi089.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-21T01:06:58.134 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:06:58.134 INFO:teuthology.orchestra.run.smithi089.stdout:Total 25 MB/s | 228 MB 00:09 2024-09-21T01:06:58.137 INFO:teuthology.orchestra.run.smithi098.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-21T01:06:58.162 INFO:teuthology.orchestra.run.smithi098.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-21T01:06:58.187 INFO:teuthology.orchestra.run.smithi098.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-21T01:06:58.208 INFO:teuthology.orchestra.run.smithi027.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 374 kB/s | 34 kB 00:00 2024-09-21T01:06:58.221 INFO:teuthology.orchestra.run.smithi098.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T01:06:58.247 INFO:teuthology.orchestra.run.smithi098.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-21T01:06:58.259 INFO:teuthology.orchestra.run.smithi027.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 15 MB/s | 3.1 MB 00:00 2024-09-21T01:06:58.272 INFO:teuthology.orchestra.run.smithi098.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T01:06:58.284 INFO:teuthology.orchestra.run.smithi027.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.0 MB/s | 154 kB 00:00 2024-09-21T01:06:58.297 INFO:teuthology.orchestra.run.smithi098.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T01:06:58.322 INFO:teuthology.orchestra.run.smithi098.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 3.6 MB/s | 581 kB 00:00 2024-09-21T01:06:58.347 INFO:teuthology.orchestra.run.smithi098.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T01:06:58.373 INFO:teuthology.orchestra.run.smithi098.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-21T01:06:58.440 INFO:teuthology.orchestra.run.smithi098.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-21T01:06:58.515 INFO:teuthology.orchestra.run.smithi098.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-21T01:06:58.549 INFO:teuthology.orchestra.run.smithi098.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-21T01:06:58.625 INFO:teuthology.orchestra.run.smithi098.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-21T01:06:58.650 INFO:teuthology.orchestra.run.smithi098.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-21T01:06:58.692 INFO:teuthology.orchestra.run.smithi098.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-21T01:06:58.701 INFO:teuthology.orchestra.run.smithi027.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-21T01:06:58.725 INFO:teuthology.orchestra.run.smithi098.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T01:06:58.759 INFO:teuthology.orchestra.run.smithi098.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-21T01:06:58.834 INFO:teuthology.orchestra.run.smithi098.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-21T01:06:58.835 INFO:teuthology.orchestra.run.smithi027.stdout:(32/157): ceph-radosgw-19.3.0-5120.g18432218.el 3.1 MB/s | 10 MB 00:03 2024-09-21T01:06:58.859 INFO:teuthology.orchestra.run.smithi098.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T01:06:58.860 INFO:teuthology.orchestra.run.smithi027.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 1.8 MB/s | 292 kB 00:00 2024-09-21T01:06:58.885 INFO:teuthology.orchestra.run.smithi098.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T01:06:58.901 INFO:teuthology.orchestra.run.smithi098.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-09-21T01:06:58.910 INFO:teuthology.orchestra.run.smithi027.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 658 kB/s | 49 kB 00:00 2024-09-21T01:06:58.977 INFO:teuthology.orchestra.run.smithi098.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-09-21T01:06:58.977 INFO:teuthology.orchestra.run.smithi027.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 141 kB/s | 16 kB 00:00 2024-09-21T01:06:58.994 INFO:teuthology.orchestra.run.smithi098.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-21T01:06:59.043 INFO:teuthology.orchestra.run.smithi098.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-21T01:06:59.070 INFO:teuthology.orchestra.run.smithi098.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 376 kB/s | 35 kB 00:00 2024-09-21T01:06:59.095 INFO:teuthology.orchestra.run.smithi098.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.8 MB/s | 250 kB 00:00 2024-09-21T01:06:59.120 INFO:teuthology.orchestra.run.smithi098.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T01:06:59.136 INFO:teuthology.orchestra.run.smithi027.stdout:(36/157): ceph-mgr-dashboard-19.3.0-5120.g18432 16 MB/s | 14 MB 00:00 2024-09-21T01:06:59.146 INFO:teuthology.orchestra.run.smithi098.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T01:06:59.161 INFO:teuthology.orchestra.run.smithi027.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 1.1 MB/s | 285 kB 00:00 2024-09-21T01:06:59.171 INFO:teuthology.orchestra.run.smithi098.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-21T01:06:59.196 INFO:teuthology.orchestra.run.smithi098.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-21T01:06:59.211 INFO:teuthology.orchestra.run.smithi027.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 245 kB/s | 18 kB 00:00 2024-09-21T01:06:59.221 INFO:teuthology.orchestra.run.smithi098.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T01:06:59.245 INFO:teuthology.orchestra.run.smithi027.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 2.9 MB/s | 784 kB 00:00 2024-09-21T01:06:59.255 INFO:teuthology.orchestra.run.smithi098.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-21T01:06:59.280 INFO:teuthology.orchestra.run.smithi098.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-09-21T01:06:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T01:06:59.354 INFO:teuthology.orchestra.run.smithi027.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 416 kB/s | 80 kB 00:00 2024-09-21T01:06:59.381 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-09-21T01:06:59.389 INFO:teuthology.orchestra.run.smithi098.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-21T01:06:59.412 INFO:teuthology.orchestra.run.smithi027.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-21T01:06:59.414 INFO:teuthology.orchestra.run.smithi098.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-21T01:06:59.440 INFO:teuthology.orchestra.run.smithi098.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-21T01:06:59.465 INFO:teuthology.orchestra.run.smithi098.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T01:06:59.479 INFO:teuthology.orchestra.run.smithi027.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-21T01:06:59.490 INFO:teuthology.orchestra.run.smithi098.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-21T01:06:59.516 INFO:teuthology.orchestra.run.smithi098.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T01:06:59.529 INFO:teuthology.orchestra.run.smithi027.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T01:06:59.536 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-09-21T01:06:59.536 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-09-21T01:06:59.549 INFO:teuthology.orchestra.run.smithi098.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.0 MB/s | 304 kB 00:00 2024-09-21T01:06:59.575 INFO:teuthology.orchestra.run.smithi098.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 983 kB/s | 58 kB 00:00 2024-09-21T01:06:59.600 INFO:teuthology.orchestra.run.smithi098.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T01:06:59.605 INFO:teuthology.orchestra.run.smithi027.stdout:(44/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 801 kB 00:00 2024-09-21T01:06:59.622 INFO:teuthology.orchestra.run.smithi027.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-21T01:06:59.625 INFO:teuthology.orchestra.run.smithi098.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T01:06:59.651 INFO:teuthology.orchestra.run.smithi098.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-09-21T01:06:59.714 INFO:teuthology.orchestra.run.smithi027.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-21T01:06:59.726 INFO:teuthology.orchestra.run.smithi098.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-21T01:06:59.772 INFO:teuthology.orchestra.run.smithi027.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-09-21T01:06:59.776 INFO:teuthology.orchestra.run.smithi098.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T01:06:59.810 INFO:teuthology.orchestra.run.smithi098.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T01:06:59.831 INFO:teuthology.orchestra.run.smithi027.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.0 MB/s | 215 kB 00:00 2024-09-21T01:06:59.835 INFO:teuthology.orchestra.run.smithi098.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-21T01:06:59.923 INFO:teuthology.orchestra.run.smithi027.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-21T01:06:59.982 INFO:teuthology.orchestra.run.smithi027.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-21T01:06:59.994 INFO:teuthology.orchestra.run.smithi098.stdout:(108/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-21T01:07:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 610 kB/s | 107 kB 00:00 2024-09-21T01:07:00.037 INFO:teuthology.orchestra.run.smithi098.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-21T01:07:00.040 INFO:teuthology.orchestra.run.smithi027.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-09-21T01:07:00.069 INFO:teuthology.orchestra.run.smithi098.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-09-21T01:07:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-09-21T01:07:00.095 INFO:teuthology.orchestra.run.smithi098.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-21T01:07:00.120 INFO:teuthology.orchestra.run.smithi098.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 593 kB/s | 49 kB 00:00 2024-09-21T01:07:00.124 INFO:teuthology.orchestra.run.smithi027.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-21T01:07:00.145 INFO:teuthology.orchestra.run.smithi098.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-21T01:07:00.166 INFO:teuthology.orchestra.run.smithi027.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-09-21T01:07:00.196 INFO:teuthology.orchestra.run.smithi098.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-21T01:07:00.208 INFO:teuthology.orchestra.run.smithi027.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-21T01:07:00.213 INFO:teuthology.orchestra.run.smithi098.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-21T01:07:00.233 INFO:teuthology.orchestra.run.smithi027.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T01:07:00.238 INFO:teuthology.orchestra.run.smithi098.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T01:07:00.255 INFO:teuthology.orchestra.run.smithi098.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 852 kB/s | 14 kB 00:00 2024-09-21T01:07:00.276 INFO:teuthology.orchestra.run.smithi027.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-21T01:07:00.289 INFO:teuthology.orchestra.run.smithi098.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-09-21T01:07:00.292 INFO:teuthology.orchestra.run.smithi027.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-21T01:07:00.314 INFO:teuthology.orchestra.run.smithi098.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T01:07:00.356 INFO:teuthology.orchestra.run.smithi098.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T01:07:00.359 INFO:teuthology.orchestra.run.smithi027.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-09-21T01:07:00.381 INFO:teuthology.orchestra.run.smithi098.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T01:07:00.401 INFO:teuthology.orchestra.run.smithi027.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-21T01:07:00.406 INFO:teuthology.orchestra.run.smithi098.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T01:07:00.418 INFO:teuthology.orchestra.run.smithi027.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T01:07:00.440 INFO:teuthology.orchestra.run.smithi098.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T01:07:00.465 INFO:teuthology.orchestra.run.smithi098.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T01:07:00.473 INFO:teuthology.orchestra.run.smithi027.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.3 MB/s | 662 kB 00:00 2024-09-21T01:07:00.491 INFO:teuthology.orchestra.run.smithi098.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-21T01:07:00.502 INFO:teuthology.orchestra.run.smithi027.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-21T01:07:00.516 INFO:teuthology.orchestra.run.smithi098.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 916 kB/s | 23 kB 00:00 2024-09-21T01:07:00.528 INFO:teuthology.orchestra.run.smithi027.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.4 MB/s | 188 kB 00:00 2024-09-21T01:07:00.541 INFO:teuthology.orchestra.run.smithi098.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-21T01:07:00.553 INFO:teuthology.orchestra.run.smithi027.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-21T01:07:00.567 INFO:teuthology.orchestra.run.smithi098.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-09-21T01:07:00.586 INFO:teuthology.orchestra.run.smithi027.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-21T01:07:00.642 INFO:teuthology.orchestra.run.smithi098.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-09-21T01:07:00.668 INFO:teuthology.orchestra.run.smithi098.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T01:07:00.693 INFO:teuthology.orchestra.run.smithi098.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T01:07:00.703 INFO:teuthology.orchestra.run.smithi027.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 28 MB/s | 4.9 MB 00:00 2024-09-21T01:07:00.718 INFO:teuthology.orchestra.run.smithi098.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T01:07:00.760 INFO:teuthology.orchestra.run.smithi098.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-09-21T01:07:00.779 INFO:teuthology.orchestra.run.smithi027.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-21T01:07:00.785 INFO:teuthology.orchestra.run.smithi098.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-21T01:07:00.811 INFO:teuthology.orchestra.run.smithi098.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T01:07:00.820 INFO:teuthology.orchestra.run.smithi027.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-21T01:07:00.844 INFO:teuthology.orchestra.run.smithi098.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-09-21T01:07:00.845 INFO:teuthology.orchestra.run.smithi027.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T01:07:00.870 INFO:teuthology.orchestra.run.smithi098.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-21T01:07:00.879 INFO:teuthology.orchestra.run.smithi027.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-21T01:07:00.895 INFO:teuthology.orchestra.run.smithi098.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-09-21T01:07:00.920 INFO:teuthology.orchestra.run.smithi098.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-21T01:07:00.921 INFO:teuthology.orchestra.run.smithi027.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T01:07:00.945 INFO:teuthology.orchestra.run.smithi098.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T01:07:00.946 INFO:teuthology.orchestra.run.smithi027.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T01:07:00.971 INFO:teuthology.orchestra.run.smithi098.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T01:07:00.988 INFO:teuthology.orchestra.run.smithi027.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T01:07:00.996 INFO:teuthology.orchestra.run.smithi098.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T01:07:01.021 INFO:teuthology.orchestra.run.smithi098.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T01:07:01.039 INFO:teuthology.orchestra.run.smithi027.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T01:07:01.046 INFO:teuthology.orchestra.run.smithi098.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T01:07:01.072 INFO:teuthology.orchestra.run.smithi098.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T01:07:01.097 INFO:teuthology.orchestra.run.smithi098.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-21T01:07:01.123 INFO:teuthology.orchestra.run.smithi098.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T01:07:01.130 INFO:teuthology.orchestra.run.smithi027.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-09-21T01:07:01.148 INFO:teuthology.orchestra.run.smithi098.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T01:07:01.156 INFO:teuthology.orchestra.run.smithi027.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-21T01:07:01.181 INFO:teuthology.orchestra.run.smithi027.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-21T01:07:01.190 INFO:teuthology.orchestra.run.smithi098.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-21T01:07:01.206 INFO:teuthology.orchestra.run.smithi027.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-21T01:07:01.248 INFO:teuthology.orchestra.run.smithi027.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-21T01:07:01.290 INFO:teuthology.orchestra.run.smithi027.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 35 kB 00:00 2024-09-21T01:07:01.353 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-09-21T01:07:01.353 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-09-21T01:07:01.399 INFO:teuthology.orchestra.run.smithi027.stdout:(83/157): postgresql-server-13.16-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-09-21T01:07:01.407 INFO:teuthology.orchestra.run.smithi098.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T01:07:01.424 INFO:teuthology.orchestra.run.smithi027.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-09-21T01:07:01.458 INFO:teuthology.orchestra.run.smithi027.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T01:07:01.475 INFO:teuthology.orchestra.run.smithi027.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T01:07:01.525 INFO:teuthology.orchestra.run.smithi027.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-09-21T01:07:01.540 INFO:teuthology.orchestra.run.smithi098.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-09-21T01:07:01.550 INFO:teuthology.orchestra.run.smithi027.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-21T01:07:01.626 INFO:teuthology.orchestra.run.smithi027.stdout:(89/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-21T01:07:01.651 INFO:teuthology.orchestra.run.smithi027.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-21T01:07:01.682 INFO:teuthology.orchestra.run.smithi098.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 289 kB/s | 41 kB 00:00 2024-09-21T01:07:01.693 INFO:teuthology.orchestra.run.smithi027.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-09-21T01:07:01.724 INFO:teuthology.orchestra.run.smithi098.stdout:(153/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.7 MB/s | 1.8 MB 00:00 2024-09-21T01:07:01.735 INFO:teuthology.orchestra.run.smithi027.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 644 kB/s | 54 kB 00:00 2024-09-21T01:07:01.760 INFO:teuthology.orchestra.run.smithi027.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-21T01:07:01.805 INFO:teuthology.orchestra.run.smithi027.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-09-21T01:07:01.805 INFO:teuthology.orchestra.run.smithi098.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2024-09-21T01:07:01.806 INFO:teuthology.orchestra.run.smithi098.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-21T01:07:01.844 INFO:teuthology.orchestra.run.smithi027.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 529 kB/s | 44 kB 00:00 2024-09-21T01:07:01.870 INFO:teuthology.orchestra.run.smithi027.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T01:07:01.911 INFO:teuthology.orchestra.run.smithi027.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 539 kB/s | 36 kB 00:00 2024-09-21T01:07:01.953 INFO:teuthology.orchestra.run.smithi027.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-21T01:07:01.979 INFO:teuthology.orchestra.run.smithi027.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-21T01:07:02.046 INFO:teuthology.orchestra.run.smithi027.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-09-21T01:07:02.062 INFO:teuthology.orchestra.run.smithi027.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-21T01:07:02.113 INFO:teuthology.orchestra.run.smithi027.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 955 kB/s | 64 kB 00:00 2024-09-21T01:07:02.155 INFO:teuthology.orchestra.run.smithi027.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-09-21T01:07:02.183 INFO:teuthology.orchestra.run.smithi098.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 2.9 MB/s | 1.4 MB 00:00 2024-09-21T01:07:02.281 INFO:teuthology.orchestra.run.smithi027.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-09-21T01:07:02.338 INFO:teuthology.orchestra.run.smithi027.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-21T01:07:02.372 INFO:teuthology.orchestra.run.smithi027.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T01:07:02.414 INFO:teuthology.orchestra.run.smithi027.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 469 kB/s | 19 kB 00:00 2024-09-21T01:07:02.456 INFO:teuthology.orchestra.run.smithi027.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-09-21T01:07:02.523 INFO:teuthology.orchestra.run.smithi027.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-09-21T01:07:02.565 INFO:teuthology.orchestra.run.smithi027.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-21T01:07:02.591 INFO:teuthology.orchestra.run.smithi027.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T01:07:02.624 INFO:teuthology.orchestra.run.smithi027.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-21T01:07:02.824 INFO:teuthology.orchestra.run.smithi027.stdout:(113/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-21T01:07:02.894 INFO:teuthology.orchestra.run.smithi027.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-09-21T01:07:02.941 INFO:teuthology.orchestra.run.smithi027.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 875 kB/s | 43 kB 00:00 2024-09-21T01:07:02.942 INFO:teuthology.orchestra.run.smithi098.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-21T01:07:02.949 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:07:02.949 INFO:teuthology.orchestra.run.smithi098.stdout:Total 19 MB/s | 228 MB 00:11 2024-09-21T01:07:02.992 INFO:teuthology.orchestra.run.smithi027.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 647 kB/s | 32 kB 00:00 2024-09-21T01:07:03.042 INFO:teuthology.orchestra.run.smithi027.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 283 kB/s | 14 kB 00:00 2024-09-21T01:07:03.067 INFO:teuthology.orchestra.run.smithi027.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-09-21T01:07:03.093 INFO:teuthology.orchestra.run.smithi027.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T01:07:03.126 INFO:teuthology.orchestra.run.smithi027.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T01:07:03.152 INFO:teuthology.orchestra.run.smithi027.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T01:07:03.177 INFO:teuthology.orchestra.run.smithi027.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T01:07:03.202 INFO:teuthology.orchestra.run.smithi027.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T01:07:03.252 INFO:teuthology.orchestra.run.smithi027.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-21T01:07:03.306 INFO:teuthology.orchestra.run.smithi027.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-09-21T01:07:03.353 INFO:teuthology.orchestra.run.smithi027.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 466 kB/s | 23 kB 00:00 2024-09-21T01:07:03.403 INFO:teuthology.orchestra.run.smithi027.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 390 kB/s | 19 kB 00:00 2024-09-21T01:07:03.455 INFO:teuthology.orchestra.run.smithi027.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 385 kB/s | 20 kB 00:00 2024-09-21T01:07:03.547 INFO:teuthology.orchestra.run.smithi027.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-21T01:07:03.589 INFO:teuthology.orchestra.run.smithi027.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-09-21T01:07:03.630 INFO:teuthology.orchestra.run.smithi027.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-09-21T01:07:03.673 INFO:teuthology.orchestra.run.smithi027.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-09-21T01:07:03.706 INFO:teuthology.orchestra.run.smithi027.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-09-21T01:07:03.748 INFO:teuthology.orchestra.run.smithi027.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-09-21T01:07:03.773 INFO:teuthology.orchestra.run.smithi027.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-09-21T01:07:03.815 INFO:teuthology.orchestra.run.smithi027.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 682 kB/s | 28 kB 00:00 2024-09-21T01:07:03.841 INFO:teuthology.orchestra.run.smithi027.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 978 kB/s | 90 kB 00:00 2024-09-21T01:07:03.883 INFO:teuthology.orchestra.run.smithi027.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-09-21T01:07:03.925 INFO:teuthology.orchestra.run.smithi027.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-21T01:07:03.967 INFO:teuthology.orchestra.run.smithi027.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 863 kB/s | 36 kB 00:00 2024-09-21T01:07:04.009 INFO:teuthology.orchestra.run.smithi027.stdout:(141/157): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-09-21T01:07:04.034 INFO:teuthology.orchestra.run.smithi027.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 979 kB/s | 188 kB 00:00 2024-09-21T01:07:04.059 INFO:teuthology.orchestra.run.smithi027.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 811 kB/s | 74 kB 00:00 2024-09-21T01:07:04.084 INFO:teuthology.orchestra.run.smithi027.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T01:07:04.109 INFO:teuthology.orchestra.run.smithi027.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 813 kB/s | 20 kB 00:00 2024-09-21T01:07:04.135 INFO:teuthology.orchestra.run.smithi027.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T01:07:04.193 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-09-21T01:07:04.230 INFO:teuthology.orchestra.run.smithi027.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-21T01:07:04.261 INFO:teuthology.orchestra.run.smithi027.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-09-21T01:07:04.311 INFO:teuthology.orchestra.run.smithi027.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-21T01:07:04.347 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-09-21T01:07:04.347 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-09-21T01:07:04.478 INFO:teuthology.orchestra.run.smithi027.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T01:07:04.653 INFO:teuthology.orchestra.run.smithi027.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-09-21T01:07:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 352 kB/s | 41 kB 00:00 2024-09-21T01:07:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.5 MB/s | 2.3 MB 00:00 2024-09-21T01:07:04.904 INFO:teuthology.orchestra.run.smithi027.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 4.2 MB/s | 1.8 MB 00:00 2024-09-21T01:07:04.929 INFO:teuthology.orchestra.run.smithi027.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.5 MB/s | 249 kB 00:00 2024-09-21T01:07:05.046 INFO:teuthology.orchestra.run.smithi027.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-21T01:07:05.238 INFO:teuthology.orchestra.run.smithi027.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-09-21T01:07:05.244 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:07:05.244 INFO:teuthology.orchestra.run.smithi027.stdout:Total 19 MB/s | 228 MB 00:11 2024-09-21T01:07:06.167 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-09-21T01:07:06.167 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-09-21T01:07:07.388 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-09-21T01:07:07.591 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-09-21T01:07:07.591 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-09-21T01:07:07.754 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-09-21T01:07:07.849 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:07:07.887 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:07:07.940 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:07:07.974 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:07:08.008 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:07:08.067 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:07:08.409 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:07:08.535 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:08.662 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:08.694 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:08.786 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:08.815 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:07:08.844 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:07:08.999 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:07:09.056 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:07:09.083 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:09.181 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:09.218 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:07:09.264 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:07:09.286 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:07:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:07:09.454 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:07:09.583 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:07:09.743 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:07:09.848 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:09.932 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:10.174 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:10.188 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-09-21T01:07:10.189 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-09-21T01:07:10.283 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:07:10.311 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:07:10.347 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:07:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:07:10.473 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:07:10.513 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:07:10.557 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:07:10.597 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:07:10.634 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:07:10.703 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:07:10.765 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:07:10.795 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:07:10.831 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:07:10.914 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:07:10.950 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:07:10.993 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:07:11.022 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:07:11.092 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:07:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:07:11.488 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:07:11.618 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:07:11.673 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:07:12.199 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:07:12.278 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:07:12.306 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:07:12.357 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:07:12.394 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:07:12.437 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:07:12.444 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-09-21T01:07:12.571 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:07:12.596 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:07:12.660 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:07:12.693 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:07:12.747 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:07:12.812 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:07:12.874 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:07:12.885 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:07:12.915 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:07:12.917 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:07:12.948 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:07:12.953 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:07:13.000 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:07:13.009 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:07:13.044 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:07:13.096 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:07:13.146 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:07:13.356 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:07:13.473 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:13.474 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:07:13.515 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:07:13.557 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:07:13.608 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:13.641 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:13.658 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:07:13.699 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:07:13.733 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:13.739 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:07:13.762 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:07:13.764 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:07:13.789 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:07:13.791 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:07:13.865 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:07:13.948 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:07:14.000 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:07:14.002 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:07:14.030 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:14.049 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:07:14.077 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:07:14.111 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:07:14.136 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:14.150 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:07:14.164 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:07:14.208 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:07:14.227 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:07:14.237 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:07:14.249 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:07:14.272 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:07:14.273 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:07:14.306 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:07:14.347 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:07:14.382 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:07:14.467 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:07:14.468 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:07:14.521 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:07:14.552 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:07:14.621 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:07:14.653 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:07:14.714 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:07:14.731 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:07:14.827 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:15.151 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:15.260 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:07:15.291 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:07:15.326 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:07:15.407 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:07:15.423 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:07:15.459 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:07:15.476 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:07:15.500 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:07:15.510 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:07:15.544 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:07:15.576 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:07:15.604 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:07:15.608 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:07:15.665 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:07:15.673 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:07:15.710 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:07:15.727 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:07:15.757 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:07:15.792 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:07:15.796 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:07:15.838 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:07:15.883 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:07:15.929 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:07:15.972 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:07:16.001 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:07:16.031 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:07:16.069 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:07:16.071 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:07:16.116 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:07:16.147 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:07:16.191 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:07:16.326 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:07:16.378 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:07:16.784 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:07:16.900 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:07:16.920 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:07:17.003 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:07:17.040 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:07:17.067 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-09-21T01:07:17.083 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:07:17.121 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:07:17.163 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:07:17.201 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T01:07:17.253 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T01:07:17.321 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:07:17.327 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T01:07:17.382 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T01:07:17.391 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:07:17.410 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T01:07:17.478 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T01:07:17.483 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:07:17.556 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:07:17.628 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:07:17.660 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:07:17.696 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:07:17.743 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:07:17.779 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:07:17.823 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:07:17.865 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:07:18.030 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T01:07:18.035 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:07:18.114 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:07:18.150 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:07:18.153 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:18.192 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:07:18.200 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:07:18.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:07:18.261 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:07:18.295 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:07:18.310 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:18.325 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:07:18.345 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:18.365 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:07:18.391 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:07:18.415 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:07:18.443 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:18.476 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:07:18.498 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:07:18.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T01:07:18.589 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:07:18.641 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:07:18.732 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:07:18.738 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T01:07:18.763 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:07:18.799 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T01:07:18.825 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:18.845 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:07:18.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:07:18.918 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:07:18.952 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:07:18.957 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:18.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:07:19.000 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:07:19.031 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:07:19.061 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:07:19.076 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T01:07:19.076 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:07:19.096 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:07:19.102 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:07:19.129 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:07:19.149 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T01:07:19.171 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:07:19.205 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:07:19.293 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:07:19.335 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:07:19.375 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:07:19.393 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:07:19.479 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:07:19.551 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:07:19.588 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:07:19.595 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T01:07:19.620 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:07:19.691 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:07:19.722 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:07:19.809 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T01:07:19.932 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:20.019 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:20.287 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:07:20.340 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:07:20.374 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:07:20.380 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T01:07:20.483 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:07:20.546 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:07:20.591 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:07:20.652 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T01:07:20.680 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:07:20.683 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T01:07:20.711 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:07:20.729 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T01:07:20.823 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T01:07:20.883 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T01:07:20.894 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:07:20.924 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:07:20.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T01:07:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T01:07:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:07:21.038 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T01:07:21.072 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T01:07:21.164 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T01:07:21.225 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T01:07:21.266 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T01:07:21.314 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T01:07:21.421 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T01:07:21.454 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T01:07:21.499 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T01:07:21.535 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T01:07:21.630 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T01:07:21.660 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:07:21.696 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T01:07:21.791 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:07:21.792 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T01:07:21.953 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:21.963 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T01:07:21.986 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:22.042 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T01:07:22.550 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:22.589 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:07:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:07:22.680 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:07:22.680 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:07:22.680 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:22.725 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:07:22.726 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T01:07:22.952 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T01:07:22.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T01:07:23.025 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:07:23.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T01:07:23.076 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T01:07:23.129 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T01:07:23.155 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:07:23.249 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:07:23.282 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:07:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T01:07:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T01:07:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T01:07:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T01:07:23.611 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:07:23.662 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:07:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T01:07:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T01:07:23.763 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:07:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T01:07:23.792 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:07:23.828 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:07:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T01:07:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T01:07:23.933 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T01:07:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T01:07:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T01:07:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T01:07:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T01:07:24.336 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:07:24.382 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:07:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T01:07:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T01:07:24.453 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:07:24.484 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:07:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T01:07:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T01:07:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T01:07:24.671 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T01:07:24.840 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T01:07:24.886 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T01:07:24.930 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T01:07:24.967 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T01:07:25.007 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T01:07:25.046 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T01:07:25.083 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T01:07:25.118 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T01:07:25.151 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T01:07:25.221 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T01:07:25.269 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T01:07:25.382 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T01:07:25.455 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T01:07:25.490 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T01:07:25.621 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T01:07:25.689 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T01:07:26.633 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T01:07:26.690 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T01:07:26.723 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T01:07:26.777 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:26.814 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:26.842 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T01:07:26.906 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T01:07:26.959 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T01:07:27.069 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T01:07:27.116 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T01:07:27.367 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T01:07:27.388 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:27.404 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T01:07:27.426 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:07:27.509 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T01:07:27.518 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:07:27.519 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:07:27.519 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:07:27.519 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:27.552 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:07:28.389 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T01:07:28.548 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T01:07:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T01:07:30.320 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T01:07:30.357 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T01:07:30.390 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T01:07:30.775 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T01:07:30.825 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:07:30.940 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T01:07:30.973 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T01:07:31.022 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T01:07:31.599 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T01:07:31.628 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:07:31.719 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T01:07:31.746 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T01:07:34.416 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-09-21T01:07:34.475 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:34.650 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:07:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:07:34.668 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:34.668 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:07:34.668 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:35.187 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T01:07:35.247 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:07:35.355 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T01:07:35.355 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T01:07:35.355 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T01:07:35.356 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:35.450 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:07:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:07:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:07:36.255 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:36.255 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:07:36.255 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:07:36.255 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:07:36.255 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:36.487 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:07:36.511 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:07:36.512 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:36.512 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:07:36.512 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:07:36.512 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:07:36.512 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:36.981 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:07:37.003 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:07:37.004 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:37.004 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:07:37.004 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:07:37.004 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:07:37.004 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:37.101 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:07:37.160 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:07:37.223 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:07:37.409 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:07:37.456 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:07:37.534 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:07:37.632 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:07:37.772 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:07:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:07:37.848 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:07:37.884 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:07:38.244 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:07:38.274 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:07:38.307 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:07:38.373 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:07:38.398 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:07:39.441 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:07:39.441 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-09-21T01:07:39.442 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:39.616 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:07:39.632 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:07:39.632 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:39.632 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:07:39.633 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:41.202 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:07:41.229 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:07:41.229 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:41.229 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:07:41.229 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:07:41.229 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:07:41.229 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:41.470 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:07:41.494 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:07:41.494 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:41.494 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:07:41.494 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:07:41.494 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:07:41.494 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:41.579 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:07:41.637 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:07:41.963 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:07:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:07:41.987 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:41.987 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:07:41.987 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:07:41.987 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:07:41.987 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:42.101 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:07:42.159 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:07:42.223 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:07:42.412 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:07:42.464 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:07:42.542 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:07:42.606 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:07:42.638 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:07:42.640 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:07:42.750 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:07:42.790 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:07:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:07:42.863 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:07:42.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:07:42.922 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:07:42.949 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:07:42.973 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:07:42.973 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:42.973 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:07:42.973 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:07:42.973 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:07:42.973 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:43.024 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:07:43.036 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:07:43.134 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:07:43.189 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:07:43.335 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:07:43.364 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:07:43.396 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:07:43.472 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:07:43.496 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:07:44.150 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:07:44.186 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:07:44.209 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:07:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:07:44.209 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:07:44.210 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:07:44.210 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:44.261 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:07:44.285 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:07:44.285 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:44.285 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:07:44.285 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:44.337 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:07:44.360 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:07:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:07:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:07:44.361 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:07:44.361 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:44.696 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:07:44.720 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:07:44.720 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:44.720 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:07:44.720 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:07:44.721 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:07:44.721 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:07:46.719 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:07:46.777 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:07:47.746 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:07:47.777 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:07:47.893 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:07:48.070 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:07:48.105 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:07:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:07:48.129 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:48.129 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:07:48.129 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:07:48.129 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:07:48.129 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:48.206 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:07:48.218 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:07:48.317 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:07:48.370 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-09-21T01:07:48.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:48.693 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:07:48.734 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:07:48.774 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:07:48.807 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:07:48.834 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:07:48.926 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:07:48.964 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T01:07:48.965 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:48.965 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T01:07:48.965 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:49.014 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:07:49.065 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:07:49.095 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:07:49.125 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:07:49.172 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:07:49.331 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:07:49.366 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:07:49.390 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:07:49.390 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:49.390 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:07:49.390 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:07:49.390 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:07:49.390 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:49.442 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:07:49.465 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:07:49.466 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:49.466 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:07:49.466 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:49.526 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:07:49.550 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:07:49.550 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:49.550 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:07:49.550 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:07:49.550 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:07:49.550 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:49.759 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:07:49.804 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:07:49.824 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:07:49.877 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:07:49.901 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:07:49.901 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:49.901 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:07:49.901 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:07:49.901 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:07:49.902 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:07:50.654 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:07:50.744 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:07:50.771 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:07:50.784 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:07:50.831 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T01:07:50.831 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:50.831 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T01:07:50.832 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:07:50.832 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T01:07:50.832 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:50.865 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:07:50.897 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:07:50.922 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:07:50.949 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:07:50.949 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:07:51.082 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:07:51.082 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:07:51.096 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:07:51.149 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T01:07:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T01:07:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:07:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T01:07:51.150 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:51.184 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:07:51.209 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:07:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:07:51.214 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:07:51.394 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:07:51.394 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:07:51.478 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:07:51.542 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:07:51.697 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:07:51.750 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T01:07:51.750 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:51.750 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T01:07:51.751 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:07:51.751 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T01:07:51.751 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:51.890 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T01:07:51.951 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T01:07:52.025 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T01:07:52.280 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T01:07:52.335 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T01:07:52.424 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T01:07:52.534 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T01:07:52.737 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T01:07:52.785 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T01:07:52.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T01:07:52.866 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T01:07:53.356 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:07:53.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T01:07:53.433 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T01:07:53.518 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T01:07:53.555 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:07:53.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:07:53.931 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:07:53.972 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:07:54.004 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:07:54.031 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:07:54.215 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:07:54.262 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:07:54.292 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:07:54.321 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:07:54.368 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:07:55.017 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:07:55.066 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:07:55.087 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:07:55.916 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:07:55.997 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:07:56.038 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:07:56.119 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:07:56.158 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:07:56.184 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:07:56.210 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:07:56.210 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:07:56.286 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:07:56.286 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:07:56.353 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:07:56.378 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:07:56.383 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:07:56.383 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:07:56.563 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:07:56.563 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:07:56.656 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:07:56.718 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:07:57.453 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T01:07:57.527 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:07:58.095 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:07:58.095 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:07:58.095 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:07:58.095 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:07:58.096 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:07:58.097 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:07:58.109 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:07:58.110 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:07:58.111 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:07:58.112 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:07:58.113 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:07:58.114 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:07:58.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:07:58.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:07:58.117 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:07:58.118 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:07:58.119 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:07:58.119 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:07:58.119 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:07:58.119 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:07:58.682 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T01:07:58.727 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:07:58.888 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T01:07:59.082 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T01:07:59.113 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:07:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T01:07:59.168 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:07:59.168 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T01:07:59.168 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:07:59.168 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T01:07:59.168 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:07:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:07:59.249 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T01:07:59.362 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T01:07:59.433 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T01:08:00.559 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T01:08:00.591 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:00.640 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.641 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.642 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.643 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:08:00.644 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:08:00.645 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:08:00.646 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:08:00.647 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:08:00.648 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:08:00.649 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:08:00.650 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:08:00.651 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:08:00.652 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-09-21T01:08:00.691 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:08:00.743 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T01:08:00.744 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:08:00.744 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T01:08:00.744 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:00.800 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:08:00.854 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T01:08:00.854 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:08:00.854 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T01:08:00.854 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:08:00.854 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T01:08:00.854 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:01.079 DEBUG:teuthology.parallel:result is None 2024-09-21T01:08:01.234 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:08:01.287 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T01:08:01.288 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T01:08:01.288 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T01:08:01.288 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:08:01.288 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T01:08:01.288 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:08:03.325 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:08:03.326 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:08:03.327 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:08:03.339 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:08:03.340 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:08:03.341 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:08:03.342 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:08:03.343 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:08:03.344 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:08:03.345 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:08:03.346 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:08:03.347 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:08:03.348 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:08:05.924 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T01:08:05.974 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T01:08:06.026 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T01:08:06.062 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T01:08:06.089 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T01:08:06.141 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:08:06.141 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:08:06.141 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-09-21T01:08:06.141 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:06.141 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:06.141 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.142 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.143 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:08:06.144 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.145 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:08:06.146 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:08:06.147 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:08:06.148 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:08:06.149 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:08:06.150 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:08:06.151 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:08:06.152 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:08:06.153 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-09-21T01:08:06.363 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T01:08:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T01:08:06.448 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T01:08:06.480 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T01:08:06.524 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T01:08:06.569 DEBUG:teuthology.parallel:result is None 2024-09-21T01:08:07.231 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:08:07.280 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:08:07.342 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T01:08:08.720 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T01:08:08.839 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T01:08:08.893 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T01:08:09.011 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T01:08:09.049 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T01:08:09.081 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T01:08:09.118 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T01:08:09.119 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:08:09.217 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T01:08:09.218 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:08:09.318 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T01:08:09.342 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T01:08:09.359 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:08:09.359 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:08:09.611 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T01:08:09.611 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:08:09.721 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T01:08:09.797 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:08:10.001 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:08:10.001 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T01:08:10.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:16.284 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T01:08:16.284 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T01:08:16.285 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T01:08:16.286 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T01:08:16.298 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T01:08:16.299 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T01:08:16.300 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T01:08:16.301 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T01:08:16.302 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T01:08:16.303 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T01:08:16.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T01:08:16.305 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T01:08:16.306 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T01:08:16.307 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T01:08:18.601 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T01:08:18.601 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:18.601 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-09-21T01:08:18.601 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:18.601 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.602 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.603 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T01:08:18.604 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T01:08:18.605 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T01:08:18.606 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T01:08:18.607 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T01:08:18.608 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T01:08:18.609 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T01:08:18.610 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T01:08:18.611 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T01:08:18.612 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T01:08:18.613 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T01:08:18.613 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T01:08:18.613 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T01:08:18.613 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T01:08:18.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:08:18.613 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-09-21T01:08:19.201 DEBUG:teuthology.parallel:result is None 2024-09-21T01:08:19.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:08:19.361 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:08:19.412 INFO:teuthology.orchestra.run.smithi027.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:08:19.413 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:08:19.413 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:08:19.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:08:19.552 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:08:19.578 INFO:teuthology.orchestra.run.smithi089.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:08:19.578 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:08:19.578 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:08:19.580 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:08:19.716 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T01:08:19.743 INFO:teuthology.orchestra.run.smithi098.stdout:19.3.0-5120.g18432218.el9 2024-09-21T01:08:19.743 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T01:08:19.743 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T01:08:19.745 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T01:08:19.745 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:08:19.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:08:19.807 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:08:19.807 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:08:19.839 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:08:19.840 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T01:08:19.872 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T01:08:19.872 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:08:19.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:08:19.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:08:20.034 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:08:20.035 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:08:20.065 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:08:20.134 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:08:20.134 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T01:08:20.172 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T01:08:20.239 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T01:08:20.240 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:08:20.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:08:20.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:08:20.407 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:08:20.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:08:20.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:08:20.498 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:08:20.498 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T01:08:20.523 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T01:08:20.589 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T01:08:20.589 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:08:20.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:08:20.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:08:20.751 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:08:20.752 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:08:20.778 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:08:20.842 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:08:20.842 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T01:08:20.868 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T01:08:20.932 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T01:08:21.035 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2}} 2024-09-21T01:08:21.035 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:08:21.036 INFO:tasks.cephadm:Cluster fsid is fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:08:21.036 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T01:08:21.036 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.27', 'mon.b': '172.21.15.89', 'mon.c': '172.21.15.98'} 2024-09-21T01:08:21.036 INFO:tasks.cephadm:First mon is mon.a on smithi027 2024-09-21T01:08:21.036 INFO:tasks.cephadm:First mgr is x 2024-09-21T01:08:21.036 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T01:08:21.036 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-09-21T01:08:21.100 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2024-09-21T01:08:21.125 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2024-09-21T01:08:21.151 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T01:08:21.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:08:21.293 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T01:08:21.431 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:08:21.432 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T01:08:21.433 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T01:08:21.433 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:21.976 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:08 /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:21.976 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:22.505 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:08 /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:22.505 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:23.035 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 01:08 /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:23.035 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:23.078 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:23.097 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T01:08:23.131 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T01:08:23.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:08:23.135 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:08:23.140 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T01:08:24.797 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:08:24.831 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:08:25.014 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:08:58.285 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-21T01:09:21.191 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T01:09:22.816 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T01:09:22.816 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:09:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T01:09:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2024-09-21T01:09:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T01:09:22.817 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-09-21T01:09:22.817 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T01:09:22.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-09-21T01:09:22.907 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2024-09-21T01:09:22.934 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2024-09-21T01:09:22.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-09-21T01:09:23.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2024-09-21T01:09:23.051 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2024-09-21T01:09:23.080 INFO:tasks.cephadm:Writing seed config... 2024-09-21T01:09:23.081 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T01:09:23.081 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T01:09:23.081 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T01:09:23.081 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T01:09:23.082 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T01:09:23.083 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T01:09:23.083 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T01:09:23.083 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T01:09:23.083 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T01:09:23.083 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T01:09:23.083 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:09:23.083 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T01:09:23.120 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = fe42ec90-77b5-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 2024-09-21T01:09:23.120 DEBUG:teuthology.orchestra.run.smithi027:mon.a> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T01:09:23.164 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T01:09:23.209 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T01:09:23.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.27 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:09:23.475 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-09-21T01:09:23.476 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.27', '--skip-admin-label'] 2024-09-21T01:09:23.476 INFO:teuthology.orchestra.run.smithi027.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T01:09:23.477 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying podman|docker is present... 2024-09-21T01:09:23.522 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.2.2 2024-09-21T01:09:23.522 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying lvm2 is present... 2024-09-21T01:09:23.523 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying time synchronization is in place... 2024-09-21T01:09:23.537 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T01:09:23.537 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T01:09:23.550 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T01:09:23.550 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T01:09:23.563 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-09-21T01:09:23.576 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-09-21T01:09:23.577 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-09-21T01:09:23.577 INFO:teuthology.orchestra.run.smithi027.stdout:Repeating the final host check... 2024-09-21T01:09:23.640 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout 5.2.2 2024-09-21T01:09:23.640 INFO:teuthology.orchestra.run.smithi027.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T01:09:23.640 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl is present 2024-09-21T01:09:23.640 INFO:teuthology.orchestra.run.smithi027.stdout:lvcreate is present 2024-09-21T01:09:23.653 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T01:09:23.653 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T01:09:23.665 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T01:09:23.666 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T01:09:23.679 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout enabled 2024-09-21T01:09:23.692 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout active 2024-09-21T01:09:23.692 INFO:teuthology.orchestra.run.smithi027.stdout:Unit chronyd.service is enabled and running 2024-09-21T01:09:23.692 INFO:teuthology.orchestra.run.smithi027.stdout:Host looks OK 2024-09-21T01:09:23.692 INFO:teuthology.orchestra.run.smithi027.stdout:Cluster fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:23.693 INFO:teuthology.orchestra.run.smithi027.stdout:Acquiring lock 140419100790400 on /run/cephadm/fe42ec90-77b5-11ef-baf5-efdc52797490.lock 2024-09-21T01:09:23.693 INFO:teuthology.orchestra.run.smithi027.stdout:Lock 140419100790400 acquired on /run/cephadm/fe42ec90-77b5-11ef-baf5-efdc52797490.lock 2024-09-21T01:09:23.693 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 3300 ... 2024-09-21T01:09:23.693 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying IP 172.21.15.27 port 6789 ... 2024-09-21T01:09:23.694 INFO:teuthology.orchestra.run.smithi027.stdout:Base mon IP(s) is [172.21.15.27:3300, 172.21.15.27:6789], mon addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T01:09:23.701 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.27 metric 100 2024-09-21T01:09:23.701 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2024-09-21T01:09:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T01:09:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T01:09:23.707 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-09-21T01:09:23.713 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T01:09:23.713 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T01:09:23.713 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T01:09:23.713 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T01:09:23.713 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T01:09:23.713 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T01:09:23.714 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-09-21T01:09:23.714 INFO:teuthology.orchestra.run.smithi027.stdout:Mon IP `172.21.15.27` is in CIDR network `172.21.0.0/20` 2024-09-21T01:09:23.714 INFO:teuthology.orchestra.run.smithi027.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T01:09:23.715 INFO:teuthology.orchestra.run.smithi027.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T01:09:23.716 INFO:teuthology.orchestra.run.smithi027.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:09:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T01:09:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T01:09:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T01:09:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T01:09:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T01:09:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T01:09:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T01:09:25.529 INFO:teuthology.orchestra.run.smithi027.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T01:09:25.529 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T01:09:25.530 INFO:teuthology.orchestra.run.smithi027.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T01:09:26.578 INFO:teuthology.orchestra.run.smithi027.stdout:stat: stdout 167 167 2024-09-21T01:09:26.578 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial keys... 2024-09-21T01:09:27.607 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQDHHO5m2K8HABAA0EGvldHucEVkvU4BwLhmzg== 2024-09-21T01:09:28.777 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQDIHO5mX3TaDBAAUwSN38Q/ba/8V/8AHyKm9w== 2024-09-21T01:09:30.120 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph-authtool: stdout AQDJHO5m1J8SEBAAbeEzNtNx65n1q88z9/11MA== 2024-09-21T01:09:30.122 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial monmap... 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool for a [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: set fsid to fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:09:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mon... 2024-09-21T01:09:32.597 INFO:teuthology.orchestra.run.smithi027.stdout:create mon.a on 2024-09-21T01:09:32.914 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T01:09:33.149 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T01:09:33.428 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490.target. 2024-09-21T01:09:33.428 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490.target. 2024-09-21T01:09:33.747 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a 2024-09-21T01:09:33.747 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T01:09:34.044 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490.target.wants/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@.service. 2024-09-21T01:09:34.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:34 smithi027 podman[30001]: 2024-09-21 01:09:34.601567388 +0000 UTC m=+0.321041094 container start 44ddfb993193af3591ee7728a35e9f3c897ce9076fe70d7b68d7a43ab1845399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T01:09:34.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:34 smithi027 bash[30001]: 44ddfb993193af3591ee7728a35e9f3c897ce9076fe70d7b68d7a43ab1845399 2024-09-21T01:09:34.706 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:09:34.706 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T01:09:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:09:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T01:09:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T01:09:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T01:09:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon to start... 2024-09-21T01:09:34.721 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mon... 2024-09-21T01:09:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:34 smithi027 systemd[1]: Started Ceph mon.a for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:09:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:34 smithi027 ceph-mon[30020]: mkfs fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:34 smithi027 ceph-mon[30020]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:09:36.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:09:36.031 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: monmap epoch 1 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: last_changed 2024-09-21T01:09:30.610244+0000 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: min_mon_release 19 (squid) 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: election_strategy: 1 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: fsmap 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: mgrmap e1: no daemons active 2024-09-21T01:09:36.032 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:35 smithi027 ceph-mon[30020]: from='client.? 172.21.15.27:0/518639697' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T01:09:36.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T01:09:36.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout id: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:36.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T01:09:36.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:36.318 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout services: 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.806428s) 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout data: 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:mon is available 2024-09-21T01:09:36.319 INFO:teuthology.orchestra.run.smithi027.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T01:09:37.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:37 smithi027 ceph-mon[30020]: from='client.? 172.21.15.27:0/1055777718' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T01:09:37.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:37 smithi027 ceph-mon[30020]: from='client.? 172.21.15.27:0/1055777718' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T01:09:37.758 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T01:09:37.759 INFO:teuthology.orchestra.run.smithi027.stdout:Generating new minimal ceph.conf... 2024-09-21T01:09:38.764 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:38 smithi027 ceph-mon[30020]: from='client.? 172.21.15.27:0/1762414233' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:09:39.036 INFO:teuthology.orchestra.run.smithi027.stdout:Restarting the monitor... 2024-09-21T01:09:39.412 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:39 smithi027 systemd[1]: Stopping Ceph mon.a for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:09:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:39 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a[30016]: 2024-09-21T01:09:39.409+0000 7fb17d200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:09:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:39 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a[30016]: 2024-09-21T01:09:39.409+0000 7fb17d200640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T01:09:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:39 smithi027 podman[30255]: 2024-09-21 01:09:39.637665256 +0000 UTC m=+0.349715272 container died 44ddfb993193af3591ee7728a35e9f3c897ce9076fe70d7b68d7a43ab1845399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T01:09:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 podman[30255]: 2024-09-21 01:09:40.024328609 +0000 UTC m=+0.736378625 container cleanup 44ddfb993193af3591ee7728a35e9f3c897ce9076fe70d7b68d7a43ab1845399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:09:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 bash[30255]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a 2024-09-21T01:09:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 podman[30267]: 2024-09-21 01:09:40.300520553 +0000 UTC m=+0.657413821 container remove 44ddfb993193af3591ee7728a35e9f3c897ce9076fe70d7b68d7a43ab1845399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T01:09:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T01:09:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 systemd[1]: Stopped Ceph mon.a for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:09:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service: Consumed 1.328s CPU time. 2024-09-21T01:09:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 systemd[1]: Starting Ceph mon.a for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:09:41.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 podman[30357]: 2024-09-21 01:09:40.695109522 +0000 UTC m=+0.030085030 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:09:41.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:40 smithi027 podman[30357]: 2024-09-21 01:09:40.826229655 +0000 UTC m=+0.161205148 container create 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T01:09:41.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 podman[30357]: 2024-09-21 01:09:41.053972396 +0000 UTC m=+0.388947891 container init 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T01:09:41.130 INFO:teuthology.orchestra.run.smithi027.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T01:09:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 podman[30357]: 2024-09-21 01:09:41.060591344 +0000 UTC m=+0.395566838 container start 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:09:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 bash[30357]: 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 systemd[1]: Started Ceph mon.a for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: pidfile_write: ignore empty --pid-file 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: load: jerasure load: lrc 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: RocksDB version: 7.9.2 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Git sha 0 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: DB SUMMARY 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: DB Session ID: X1TB650M859ILN1PG70G 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: CURRENT file: CURRENT 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T01:09:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94860 ; 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.error_if_exists: 0 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.create_if_missing: 0 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.paranoid_checks: 1 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.env: 0x5611a84b2d60 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.info_log: 0x5611e8277320 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.statistics: (nil) 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.use_fsync: 0 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_log_file_size: 0 2024-09-21T01:09:41.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T01:09:41.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T01:09:41.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.allow_fallocate: 1 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.use_direct_reads: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.db_log_dir: 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.wal_dir: 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T01:09:41.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.write_buffer_manager: 0x5611e827bd60 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T01:09:41.436 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.unordered_write: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.row_cache: None 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.wal_filter: None 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.two_write_queues: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.wal_compression: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.atomic_flush: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T01:09:41.437 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.log_readahead_size: 0 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_background_jobs: 2 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_background_compactions: -1 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_subcompactions: 1 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T01:09:41.438 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_open_files: -1 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_background_flushes: -1 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Compression algorithms supported: 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kZSTD supported: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kXpressCompression supported: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kBZip2Compression supported: 0 2024-09-21T01:09:41.439 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kLZ4Compression supported: 1 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kZlibCompression supported: 1 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: kSnappyCompression supported: 1 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.merge_operator: 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_filter: None 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T01:09:41.440 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5611e8276f60) 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_type: 0 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_index_type: 0 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_shortening: 1 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: checksum: 4 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: no_block_cache: 0 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache: 0x5611e423b610 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_options: 2024-09-21T01:09:41.441 INFO:journalctl@ceph.mon.a.smithi027.stdout: capacity : 536870912 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_shard_bits : 4 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: strict_capacity_limit : 0 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_cache_compressed: (nil) 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: persistent_cache: (nil) 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size: 4096 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_size_deviation: 10 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_restart_interval: 16 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: index_block_restart_interval: 1 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: metadata_block_size: 4096 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: partition_filters: 0 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: use_delta_encoding: 1 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: filter_policy: bloomfilter 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: whole_key_filtering: 1 2024-09-21T01:09:41.442 INFO:journalctl@ceph.mon.a.smithi027.stdout: verify_compression: 0 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: format_version: 5 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_index_compression: 1 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: block_align: 0 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: max_auto_readahead_size: 262144 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: prepopulate_block_cache: 0 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: initial_auto_readahead_size: 8192 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression: NoCompression 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.num_levels: 7 2024-09-21T01:09:41.443 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T01:09:41.444 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T01:09:41.445 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T01:09:41.446 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.inplace_update_support: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.bloom_locality: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.max_successive_merges: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T01:09:41.447 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.ttl: 2592000 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.enable_blob_files: false 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.min_blob_size: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T01:09:41.448 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dcbe9692-a147-478f-8361-2f5c4a16533c 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726880981184958, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726880981187120, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 89150, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11607, "raw_average_key_size": 47, "raw_value_size": 82550, "raw_average_value_size": 341, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726880981, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dcbe9692-a147-478f-8361-2f5c4a16533c", "db_session_id": "X1TB650M859ILN1PG70G", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726880981187309, "job": 1, "event": "recovery_finished"} 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5611e829ca80 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: DB pointer 0x5611e903e000 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** DB Stats ** 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:09:41.449 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: L0 2/0 90.83 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Sum 2/0 90.83 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** Compaction Stats [default] ** 2024-09-21T01:09:41.450 INFO:journalctl@ceph.mon.a.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 61.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Cumulative compaction: 0.00 GB write, 13.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Interval compaction: 0.00 GB write, 13.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache BinnedLRUCache@0x5611e423b610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-21T01:09:41.451 INFO:journalctl@ceph.mon.a.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: starting mon.a rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???) e1 preinit fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).mds e1 new map 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).mds e1 print_map 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: e1 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: btime 2024-09-21T01:09:34:736922+0000 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: legacy client fscid: -1 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout: No filesystems configured 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:09:41.452 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: monmap epoch 1 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: last_changed 2024-09-21T01:09:30.610244+0000 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: election_strategy: 1 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: fsmap 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T01:09:41.453 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:41 smithi027 ceph-mon[30375]: mgrmap e1: no daemons active 2024-09-21T01:09:42.390 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T01:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:09:42.417 INFO:teuthology.orchestra.run.smithi027.stdout:Creating mgr... 2024-09-21T01:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T01:09:42.418 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T01:09:42.771 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x 2024-09-21T01:09:42.771 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T01:09:43.014 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490.target.wants/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@.service. 2024-09-21T01:09:43.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:42 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2033503937' entity='client.admin' 2024-09-21T01:09:43.360 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 systemd[1]: Starting Ceph mgr.x for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:09:43.623 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 podman[30586]: 2024-09-21 01:09:43.264181288 +0000 UTC m=+0.027904525 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:09:43.623 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 podman[30586]: 2024-09-21 01:09:43.378583501 +0000 UTC m=+0.142306736 container create a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T01:09:43.623 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 podman[30586]: 2024-09-21 01:09:43.622882143 +0000 UTC m=+0.386605379 container init a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:09:43.784 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:09:43.784 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T01:09:43.798 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:09:43.798 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T01:09:43.798 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T01:09:43.798 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T01:09:43.816 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:09:43.816 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T01:09:43.830 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:09:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T01:09:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T01:09:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T01:09:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr to start... 2024-09-21T01:09:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr... 2024-09-21T01:09:43.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 podman[30586]: 2024-09-21 01:09:43.629294351 +0000 UTC m=+0.393017598 container start a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:09:43.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 bash[30586]: a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 2024-09-21T01:09:43.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:43 smithi027 systemd[1]: Started Ceph mgr.x for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:09:44.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:44 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2474707283' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:09:45.095 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:45.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:09:45.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:09:45.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:09:45.101 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:09:45.102 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:09:45.103 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:09:34:736922+0000", 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.104 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:09:34.738008+0000", 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:45.105 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:09:45.106 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:09:45.106 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (1/15)... 2024-09-21T01:09:48.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:47 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/137162645' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:09:48.460 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:09:48.461 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:09:48.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:09:48.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:09:48.462 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:09:48.465 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:48.466 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:09:34:736922+0000", 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:48.467 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:09:34.738008+0000", 2024-09-21T01:09:48.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:09:48.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:48.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:09:48.468 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:09:48.468 INFO:teuthology.orchestra.run.smithi027.stdout:mgr not available, waiting (2/15)... 2024-09-21T01:09:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: Activating manager daemon x 2024-09-21T01:09:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: mgrmap e2: x(active, starting, since 0.00499879s) 2024-09-21T01:09:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:09:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' 2024-09-21T01:09:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:50 smithi027 ceph-mon[30375]: from='mgr.14100 172.21.15.27:0/3690849077' entity='mgr.x' 2024-09-21T01:09:51.731 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:51.731 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:09:51.731 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 0 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T01:09:51.732 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T01:09:51.733 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T01:09:51.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T01:09:51.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T01:09:51.736 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T01:09:34:736922+0000", 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T01:09:51.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ], 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T01:09:34.738008+0000", 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout }, 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T01:09:51.738 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:09:51.739 INFO:teuthology.orchestra.run.smithi027.stdout:mgr is available 2024-09-21T01:09:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:52 smithi027 ceph-mon[30375]: mgrmap e3: x(active, since 1.00969s) 2024-09-21T01:09:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:52 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3304899686' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T01:09:53.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:53 smithi027 ceph-mon[30375]: mgrmap e4: x(active, since 2s) 2024-09-21T01:09:53.155 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:53 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2272390817' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout fsid = fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 2024-09-21T01:09:53.162 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T01:09:53.163 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T01:09:53.163 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T01:09:53.163 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T01:09:53.163 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T01:09:53.163 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T01:09:53.163 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling cephadm module... 2024-09-21T01:09:54.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:54 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/246134669' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T01:09:54.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:54 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ignoring --setuser ceph since I am not root 2024-09-21T01:09:54.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:09:54 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ignoring --setgroup ceph since I am not root 2024-09-21T01:09:55.272 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:55 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/246134669' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T01:09:55.272 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:55 smithi027 ceph-mon[30375]: mgrmap e5: x(active, since 4s) 2024-09-21T01:09:56.397 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:09:56 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1011777253' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T01:09:56.726 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-09-21T01:09:56.727 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 5... 2024-09-21T01:10:00.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: Active manager daemon x restarted 2024-09-21T01:10:00.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: Activating manager daemon x 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: mgrmap e6: x(active, starting, since 0.00413383s) 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:10:00.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:00 smithi027 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T01:10:02.182 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:10:02.182 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T01:10:02.182 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T01:10:02.182 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:10:02.182 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 5 is available 2024-09-21T01:10:02.182 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying orchestrator module is enabled... 2024-09-21T01:10:02.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: mgrmap e7: x(active, since 1.00738s) 2024-09-21T01:10:02.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:02.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:02.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: Found migration_current of "None". Setting to last migration. 2024-09-21T01:10:02.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:02.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:02.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:02.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:02.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:10:02.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:02 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:10:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-mon[30375]: mgrmap e8: x(active, since 2s) 2024-09-21T01:10:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3170916674' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T01:10:03.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:10:03.484+0000 7fcb8b200640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:10:03] ENGINE Error in HTTPServer.serve 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: Traceback (most recent call last): 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._connections.run(self.expiration_interval) 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._run(expiration_interval) 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: s = self.context.wrap_socket( 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: return self.sslsocket_class._create( 2024-09-21T01:10:03.932 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:10:03.933 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self.do_handshake() 2024-09-21T01:10:03.933 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:10:03.933 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._sslobj.do_handshake() 2024-09-21T01:10:03.933 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:10:03.933 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:10:04.596 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T01:10:04.596 INFO:teuthology.orchestra.run.smithi027.stdout:Setting orchestrator backend to cephadm... 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:03] ENGINE Bus STARTING 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:03] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:03] ENGINE Error in HTTPServer.serve 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:10:04.867 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3170916674' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T01:10:04.868 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: mgrmap e9: x(active, since 3s) 2024-09-21T01:10:04.869 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:03] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T01:10:04.869 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:03] ENGINE Bus STARTED 2024-09-21T01:10:04.869 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:04 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:06 smithi027 ceph-mon[30375]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:06 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:06 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:07.586 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:07 smithi027 ceph-mon[30375]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:07.591 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T01:10:07.592 INFO:teuthology.orchestra.run.smithi027.stdout:Generating ssh key... 2024-09-21T01:10:08.681 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: Generating public/private rsa key pair. 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: Your identification has been saved in /tmp/tmpbokwdmec/key 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: Your public key has been saved in /tmp/tmpbokwdmec/key.pub 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: The key fingerprint is: 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: SHA256:EZ/AidkKlcVW7aUl7EtANB+dN4ElH24Rh0lUu79vNGA ceph-fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: The key's randomart image is: 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: +---[RSA 3072]----+ 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | ..O+==o.==@=| 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | . + *+.++o@.*| 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | . o. o+.= *.| 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | . . =E. .| 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | S ..... | 2024-09-21T01:10:08.682 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | . .o| 2024-09-21T01:10:08.683 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | .o| 2024-09-21T01:10:08.683 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | o| 2024-09-21T01:10:08.683 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: | oo| 2024-09-21T01:10:08.683 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: +----[SHA256]-----+ 2024-09-21T01:10:09.803 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:09 smithi027 ceph-mon[30375]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:09.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:09 smithi027 ceph-mon[30375]: Generating ssh key... 2024-09-21T01:10:09.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:09 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:09.804 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:09 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:10.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:10 smithi027 ceph-mon[30375]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:10.737 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:10:10.738 INFO:teuthology.orchestra.run.smithi027.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T01:10:10.738 INFO:teuthology.orchestra.run.smithi027.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T01:10:10.745 INFO:teuthology.orchestra.run.smithi027.stdout:Adding host smithi027... 2024-09-21T01:10:12.631 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:12 smithi027 ceph-mon[30375]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:13.632 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:13 smithi027 ceph-mon[30375]: Deploying cephadm binary to smithi027 2024-09-21T01:10:16.579 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Added host 'smithi027' with addr '172.21.15.27' 2024-09-21T01:10:16.579 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mon service... 2024-09-21T01:10:16.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:16 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:16.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:16 smithi027 ceph-mon[30375]: Added host smithi027 2024-09-21T01:10:16.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:16 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:18.819 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T01:10:18.820 INFO:teuthology.orchestra.run.smithi027.stdout:Deploying unmanaged mgr service... 2024-09-21T01:10:19.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:18 smithi027 ceph-mon[30375]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:19.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:18 smithi027 ceph-mon[30375]: Saving service mon spec with placement count:5 2024-09-21T01:10:19.043 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:18 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:20.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:20 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:20.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:20 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:20.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:20 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:21.060 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T01:10:21.589 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:21 smithi027 ceph-mon[30375]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:21.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:21 smithi027 ceph-mon[30375]: Saving service mgr spec with placement count:2 2024-09-21T01:10:21.590 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:21 smithi027 ceph-mon[30375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:10:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:22 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3427947097' entity='client.admin' 2024-09-21T01:10:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:22 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:24.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:23 smithi027 ceph-mon[30375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:10:24.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:23 smithi027 ceph-mon[30375]: from='mgr.14118 172.21.15.27:0/371355203' entity='mgr.x' 2024-09-21T01:10:24.078 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:23 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/262000065' entity='client.admin' 2024-09-21T01:10:24.476 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling the dashboard module... 2024-09-21T01:10:26.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:25 smithi027 ceph-mon[30375]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:10:26.007 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:25 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1006012595' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T01:10:26.407 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ignoring --setuser ceph since I am not root 2024-09-21T01:10:26.407 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ignoring --setgroup ceph since I am not root 2024-09-21T01:10:27.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:26 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1006012595' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T01:10:27.042 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:26 smithi027 ceph-mon[30375]: mgrmap e10: x(active, since 25s) 2024-09-21T01:10:28.156 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:27 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3598202017' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T01:10:28.597 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:10:28.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T01:10:28.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T01:10:28.598 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T01:10:28.599 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T01:10:28.599 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:10:28.599 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for the mgr to restart... 2024-09-21T01:10:28.599 INFO:teuthology.orchestra.run.smithi027.stdout:Waiting for mgr epoch 10... 2024-09-21T01:10:32.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: Active manager daemon x restarted 2024-09-21T01:10:32.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: Activating manager daemon x 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: mgrmap e11: x(active, starting, since 0.00452437s) 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:10:32.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:32 smithi027 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T01:10:33.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:33.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:10:33.797 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:10:33.798 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:33 smithi027 ceph-mon[30375]: mgrmap e12: x(active, since 1.00823s) 2024-09-21T01:10:34.128 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout { 2024-09-21T01:10:34.129 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T01:10:34.129 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T01:10:34.129 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout } 2024-09-21T01:10:34.129 INFO:teuthology.orchestra.run.smithi027.stdout:mgr epoch 10 is available 2024-09-21T01:10:34.129 INFO:teuthology.orchestra.run.smithi027.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T01:10:34.383 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:10:34.171+0000 7f3195e00640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:10:34] ENGINE Error in HTTPServer.serve 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: Traceback (most recent call last): 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._connections.run(self.expiration_interval) 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._run(expiration_interval) 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: s = self.context.wrap_socket( 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:10:34.384 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: return self.sslsocket_class._create( 2024-09-21T01:10:34.385 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:10:34.385 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self.do_handshake() 2024-09-21T01:10:34.385 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:10:34.385 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._sslobj.do_handshake() 2024-09-21T01:10:34.385 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:10:34.385 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:10:34.666 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:34 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:34] ENGINE Bus STARTING 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:35 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:34] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:35 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:34] ENGINE Error in HTTPServer.serve 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:10:35.681 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:35 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:34] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:35 smithi027 ceph-mon[30375]: [21/Sep/2024:01:10:34] ENGINE Bus STARTED 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:35 smithi027 ceph-mon[30375]: mgrmap e13: x(active, since 2s) 2024-09-21T01:10:35.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:35 smithi027 ceph-mon[30375]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:36.245 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUQ4w/W2YrTTNxWj7kNMyEvADpEqAwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAxMTAzNVoXDTM0\nMDkyMjAxMTAzNVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjI3MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEApYbQU6WtYWiEV2kJCeWAbGWE1uewKkhNKzAh\nVmC87HEvUkp1ckfTQd5UCxtfoTW9BAiySITYUE6vBKD0ybTL0qCIE5WcDb7IrcYa\navRNssTsvEf1hZySICpjRwV8UdFAvc0q1bzKVG1U4biJGgN2rEorf/5SdXEpyYYo\niYPxhQiMYhmoN5VARIyuup+WFKgl4kelQG8WU+QEDJ1qXv7XhbXsGf3phKs4BBJ5\nciiq05+ukyZeixiXxrML+OsujWrxbKWsVRO4dnKdgPL5jKLg5Ig/+TSFxllnCOYw\nbSpV18NnF4cqEKEjI2tYEUIZX9vcv1D/HNTD2MA+EupAKI4hzFnfikzZxiZL8vjz\n8AyjWrLomZRTbZefGb7MhEmWCRQVY0knFGMjK+SwozFd5wrdCXMEP+HDjz1Aid8x\nPD9lVipKp91eLTDmiwCLiNCBuMWwrq7nhjSUaj+aOozqyaPUP1qQg5wpekcRr+qD\nrtdG2xaxwXhQqh1/Z1/hkigM+/XLn+bjIcJ2E3xFguvahQ6j5wfIrX/X/cjeegNz\nC04Iv2tPf2qdJf0kZ3NxkHsqfRtYjmMj1kP8fWRltROCg7zhKZBtHbHuyT903RUE\noroGq86oo9doo38/HTFRxpTbpzu/fcI8+eP9vk39j3bzou1aV3g39kUQDhbgkbFj\nRp+98+kCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDI3ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPGzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQC1\n49Oe0Ush3m4Qft8hStIXb50S4ov0OfJ45NJTuWlXxb+2/Z5CbqUKP2SAKSF+0v9T\nWuFfkoZeH9s1TPC2PtHhJ9wIosEmBmclXGo73dgKnd/JB0jA6xOvB1DA6x3xa5YF\nWgCj6ygKLThseGbuuVCXfeF3kA8aDDAsR1jbPKFVxEhSuk0zy4WzpZN72TgN7EKr\nXo/ufM9/V47gCZhIzYNBEp5GwSAIc8nX27d45HZE5zxkXPJsVnXWdr1BOz6jKqeI\n/+FHQsNWhqwRn+9IfC1HRkdRjFp39N74XBrr/xrS+VttJ4AYs5ng9WY0v0hQtnYU\nsCBVgKA14+gLT2O0JtFI6Orcgz7MNv2DiJQzESVY/C2x9ukwtmFgZAkWT+6CQxDl\nDfQXgaz/fUdL1QRO83RyCtp8/a5ciRsD08NHJyCRR+TNxOuxmSNRg3yL3MPmhIbD\n5Z+ofHE6MTLQ5rZwn25XNi6puvA4dD82BqmLEFU7kmtdtdIoR/zDkkg4ujhVhgQc\nNVECGBlaCSP9ocqb5JWJxmeGJsgbU4PYaMjV2Z++HElsyHvp5D7S5btiIhHIi/TA\n1MHh0cjEd+ahwVl0lZ5XUeV4y8h4mktwpm9XECCdL803GeaoeeWhpt0WZ6azY19I\n0lgk9eOG9/jkuPMnkkC0QxCcO6oVox5GVa3BYuUTMA==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEApYbQU6WtYWiEV2kJCeWAbGWE1uewKkhNKzAhVmC87HEvUkp1\nckfTQd5UCxtfoTW9BAiySITYUE6vBKD0ybTL0qCIE5WcDb7IrcYaavRNssTsvEf1\nhZySICpjRwV8UdFAvc0q1bzKVG1U4biJGgN2rEorf/5SdXEpyYYoiYPxhQiMYhmo\nN5VARIyuup+WFKgl4kelQG8WU+QEDJ1qXv7XhbXsGf3phKs4BBJ5ciiq05+ukyZe\nixiXxrML+OsujWrxbKWsVRO4dnKdgPL5jKLg5Ig/+TSFxllnCOYwbSpV18NnF4cq\nEKEjI2tYEUIZX9vcv1D/HNTD2MA+EupAKI4hzFnfikzZxiZL8vjz8AyjWrLomZRT\nbZefGb7MhEmWCRQVY0knFGMjK+SwozFd5wrdCXMEP+HDjz1Aid8xPD9lVipKp91e\nLTDmiwCLiNCBuMWwrq7nhjSUaj+aOozqyaPUP1qQg5wpekcRr+qDrtdG2xaxwXhQ\nqh1/Z1/hkigM+/XLn+bjIcJ2E3xFguvahQ6j5wfIrX/X/cjeegNzC04Iv2tPf2qd\nJf0kZ3NxkHsqfRtYjmMj1kP8fWRltROCg7zhKZBtHbHuyT903RUEoroGq86oo9do\no38/HTFRxpTbpzu/fcI8+eP9vk39j3bzou1aV3g39kUQDhbgkbFjRp+98+kCAwEA\nAQKCAgAoyzZZfxnhSUhgYH79ROXnEQfxjM0SRppaG/UDXDkDTou6jKYwQXpTn+73\ng8oIKX3JWjlR/Qdwl0uz6NUbRub6pFyU8wiwLIY5qXqGRGpHthef+V4NjT5IFho8\nfSIb6eW6QS+e2HXG4+EI9hk344WiRW8+DpgU4IxLYbho1680uZwbazZVaH+bxU4k\nqULsGVqozz5vSFFo4GeNEliOS+Eiz2h+KWEdNLNL8Z+jnlknaZG1Kz4eqrNxIVxA\nfvmMmcAhJh14JDGQY4PkrT0KR3QEbn3RtXuInCzz+QMTTF02nS4xkrqvuX628nQU\nYMVvJfjb36LX4KnZp9tEfx3LwOecFwX1UbNeFBPCLbHyv84OzeoqLvol17vSUkm2\nIAN8FLwLiBConHAzNZUAp2YLegYGRc5ZdGLZzZVNBH8XQ5TK+2bO+W1WlstSYfLn\nWszzdi/NlxZHVKbRpc4sySJRDUYVYl1JKXzZLlAHDKOSRUL6fJiQCsizpmHqvjKV\nNr2wE8slCdA/DHWFS+dJ3/oYLs7ZAYvvUfleslY/Dj3mcQfpa4xTW/BEnCwMsUTo\nx5cteom7JLLXudMx9SsfoWsMJ2Wrg54ublQ3GAuni82Pu5iPn/GZ3lxKgYHHEnFm\n0LmKh/579VYNCa29mnAG/b0EkkamNJK1oosLP2K0y1d0IGHQPwKCAQEA3akpckUH\nfPsmkYfCEarOxIOVfPX/AS15s3VqqLJFUf5hLTmGwN/t/mM1Njpi00ZLqPn7WTHR\n19FK+zthwzcCkdUVF94n1OCiSIJaDiK+Q9DrM03MIO8OJg07GWRv5GyM0xd7cz9j\nXW496omqvTpFOZwXJx/58d9Db4J7hQGvOhkgiyVwbBHLL5Cg3ghWhjxnJ3VeQK9U\nB02aKY6nYIuyGXjwBoXBoVkqu7VXwD/EbbsLUVc6Ol2ZF+oYSO2zfcXc+eOlZelA\naHIvHzcw93cIKU8qyT3p9pLi2Oj0PtaDAmbSnCHCrkA2Eh1v8TGUga87HUPUfbtk\npEcg+fbWjiyDswKCAQEAvytttKIkNnQt6dd171sfmvoZSc1V1PwrICiZCtxMQRPT\nfBw0Z7lWU+rKtGGshbyiQrYn7bXyK2oNn8vKo+6BfmvR1ad0jW/GmwamV/tDNW8C\nw+8GRr5j1SU0BUNhuABCH1QDkyoIiMKr/fxMwqwqmRyIt75ExR+AHMMMH44EVL+d\nsBb6PmzwuuLNwwqlAzWA/tBN3H11HYEY923eoYWq0CRqmWB1mq8Q61lRO++YGmnq\n6VGbfvuP0iLTC4H3feYdvTEdHsB+Vc5PWECqMO9uwzcQz57d/e0NhrUeeh7coyTz\ndYOnHVnQ2vubp6UgFKYaPF1ZLZNiI9V8IK/ZFbrL8wKCAQBYT1Lh4bqQAodlJ48b\n9s8V386o0GDeGT4JT1hFCXEwr4YLvIWBRjXNSb0jc7N3tOPj2GRZLs4Ji6qx2QCk\nA8CUVS5oKev4ld5RwQ0b4J+o6Q0rjW0sGdGrO042WZtyiAS6/PoD+lceHBOy41IQ\nvaxivVtEDStJP8fgHz4SyY8EOIqMpUfa5mPq+VX935iJNntqbwoz/yYdGr+2msz4\nW++C0nneP1/bZmjuC9uADme1x/VEXNon8q5RpsQyGfGOxEu7iwSUS4+LjSJU72ok\nzVmFC7NiJbaWtAKIpJbaih/7zvdNmO20+oQQR4YXs8cUu7lzeVipFwWLpYLv03W7\nguSdAoIBAGSYP7RLle6pl/dzD3YSZ0Ev1drGGCJpR3A2rMHQRrf/7MtkNBdYqZqi\n9GOS+iutpVyhacVmdrZtqhmqLUmyGaYiqlpZMbWzWbj9+M5XQ3h8n+Hmh1D2oqXo\nW8VhPayLRkAh/3eTKMZ+We7esq3F1uyW3UKC9sEb4FhCMsDwBEtsQVgF30y2Gr7c\n1hWKJmbX4rczeaVKxzPtWOvqfrpUEhxQ7V8kKP5qnOabK2n0m/9aOOjprn9VdMrI\n1HE7xLYVPxTtITurY09XrtzruiLh+1DySwBVws75zKJT8GOmlcerjAkMbAW/220/\n9Rp0g7sZ5LhbDrAy2/OrRRQWNI+Aqg0CggEAUbre2GbojJQvH+dn6JjA2S5jlMPk\n/MmXWRjTDVD1+bW1ZaoXlntX+f+vJ2EYTrK/DaqH8zXLyCUGjhFpafWv9YAHcJNH\nWUfxbNufobYkN32tGlFYH2nT0ehZTyvFBWj3kIPNoN7wMZCL97YR4rXo5eArnL/t\n28jY/0wKxs91FHQH2jp2lNu0hjvIZZUJri64MEH7jRa2rj80XZjZCO+ONaItH0i5\ngOnaZmacKzbAC3K7yBox/JJ/6cCi6mgl8/lpoyLxnNR0iVtGdysb0/elnHX+HSqJ\nDyx+eYzMls6jg5kDAtw8TWNQ1/DGjx4c9+/vpV5qKywQMERaLC/C8HJm6Q==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T01:10:36.809 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:36 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:37.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:37 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:37.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:37 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:37.978 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T01:10:38.846 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:38 smithi027 ceph-mon[30375]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:40.448 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T01:10:40.448 INFO:teuthology.orchestra.run.smithi027.stdout:Creating initial admin user... 2024-09-21T01:10:40.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:40 smithi027 ceph-mon[30375]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:40.962 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:40 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:42.165 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$sqKyx7.zlYdc1Pkl3XzSGuQKGBcXYmNzsSlYV7fqsultfP25nnLOO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726881041, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T01:10:42.165 INFO:teuthology.orchestra.run.smithi027.stdout:Fetching dashboard port number... 2024-09-21T01:10:42.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:42 smithi027 ceph-mon[30375]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:42.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:42 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:43.855 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:43 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1333304243' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T01:10:43.860 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T01:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T01:10:43.879 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout disabled 2024-09-21T01:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T01:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout:systemctl: stdout inactive 2024-09-21T01:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout:firewalld.service is not enabled 2024-09-21T01:10:43.893 INFO:teuthology.orchestra.run.smithi027.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T01:10:43.895 INFO:teuthology.orchestra.run.smithi027.stdout:Ceph Dashboard is now available at: 2024-09-21T01:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: URL: https://smithi027.front.sepia.ceph.com:8443/ 2024-09-21T01:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: User: admin 2024-09-21T01:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: Password: f3audxjfs1 2024-09-21T01:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:43.896 INFO:teuthology.orchestra.run.smithi027.stdout:Saving cluster configuration to /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config directory 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.440 INFO:teuthology.orchestra.run.smithi027.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: ceph telemetry on 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout:For more information see: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:45.441 INFO:teuthology.orchestra.run.smithi027.stdout:Bootstrap complete. 2024-09-21T01:10:45.513 INFO:tasks.cephadm:Fetching config... 2024-09-21T01:10:45.514 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:10:45.514 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T01:10:45.551 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T01:10:45.551 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:10:45.552 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T01:10:45.627 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T01:10:45.628 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:10:45.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T01:10:45.693 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:45 smithi027 ceph-mon[30375]: mgrmap e14: x(active, since 12s) 2024-09-21T01:10:45.693 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:45 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4015552040' entity='client.admin' 2024-09-21T01:10:45.725 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T01:10:45.726 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:10:45.726 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T01:10:45.801 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T01:10:45.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:10:45.921 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:10:45.947 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:10:45.991 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:10:46.003 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T01:10:46.047 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFswjOxHkaXs9rV3Wp8vcW7TKuD6T7X2J35lKKZxjuUD3D3+YsHq1OMCLLXNGbJx6ZOPgJxRUqkTshIvAOjlIfBBreAxya2o91q/qhAZdmWFzmKrkFGf+nSxG0PaLYMpzQ3wFY/zXZ4nQfWbte2VhE1crxT7/Exz64WDdJ/BnUFDy54LxhQVHC587tbEaW0+qJE0epKZBppkO0/v2iMgsB6panoQ4dLsHSHnvwEmtP8cZglz4aRc5PCME+NDQW2aF5YZL5CRA4absthObCEQnMn/r9Xub6YzDOQfRG9O8NWARPhlRHLgsrmLuGN2vvV8X2CaQNYbz6oqPybR6kD1sJgdrfJLblX5Omaq0Kwlwho6SSiKWWu92BoJPCRRc7sSf1uT7ocpSGPUcy/ipEQl91gFJDTonuDODGVnc9j1nPoLBBtvysKqrkVQsxihID5hP55dqxsSMzjVAFflarEJT/M3pvR1DHI9Lp0iF9DEKxenwb/cy8xaAXSHcO7v/8LWM= ceph-fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:10:46.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T01:10:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:10:48.010 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T01:10:48.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T01:10:48.577 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:10:48.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:48 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3775944189' entity='client.admin' 2024-09-21T01:10:50.434 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi089 2024-09-21T01:10:50.435 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:10:50.435 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.conf 2024-09-21T01:10:50.454 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:10:50.454 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:10:50.511 INFO:tasks.cephadm:Adding host smithi089 to orchestrator... 2024-09-21T01:10:50.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch host add smithi089 2024-09-21T01:10:50.766 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:50 smithi027 ceph-mon[30375]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:50.766 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:50 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:51.237 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:10:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:10:52.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:52 smithi027 ceph-mon[30375]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:53 smithi027 ceph-mon[30375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:10:54.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:54 smithi027 ceph-mon[30375]: Deploying cephadm binary to smithi089 2024-09-21T01:10:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:55 smithi027 ceph-mon[30375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:10:56.089 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi089' with addr '172.21.15.89' 2024-09-21T01:10:56.837 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T01:10:57.211 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:10:57.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:57 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:57.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:57 smithi027 ceph-mon[30375]: Added host smithi089 2024-09-21T01:10:57.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:57 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:10:57.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:57 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:57.350 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:57 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:58.067 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:10:58.067 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}] 2024-09-21T01:10:58.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:58 smithi027 ceph-mon[30375]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:10:58.415 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:58 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:10:58.779 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2024-09-21T01:10:58.779 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:10:58.779 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2024-09-21T01:10:58.798 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:10:58.798 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:10:58.857 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2024-09-21T01:10:58.857 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch host add smithi098 2024-09-21T01:10:59.239 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:10:59.381 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:10:59 smithi027 ceph-mon[30375]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:11:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:00 smithi027 ceph-mon[30375]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:01.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:01 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:02 smithi027 ceph-mon[30375]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:02 smithi027 ceph-mon[30375]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:02 smithi027 ceph-mon[30375]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:02 smithi027 ceph-mon[30375]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:02 smithi027 ceph-mon[30375]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:03.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:03 smithi027 ceph-mon[30375]: Deploying cephadm binary to smithi098 2024-09-21T01:11:03.891 INFO:teuthology.orchestra.run.smithi027.stdout:Added host 'smithi098' with addr '172.21.15.98' 2024-09-21T01:11:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:04 smithi027 ceph-mon[30375]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:04 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:04 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:04.568 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T01:11:04.942 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:05.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:05 smithi027 ceph-mon[30375]: Added host smithi098 2024-09-21T01:11:05.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:05 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:05.384 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:05 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:05.784 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:11:05.784 INFO:teuthology.orchestra.run.smithi027.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2024-09-21T01:11:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:06 smithi027 ceph-mon[30375]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:06 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:06.601 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T01:11:06.601 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T01:11:06.975 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:07.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:07 smithi027 ceph-mon[30375]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:11:08.111 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/873218536' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:08 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:08.799 INFO:tasks.cephadm:Adding mon.a on smithi027 2024-09-21T01:11:08.799 INFO:tasks.cephadm:Adding mon.b on smithi089 2024-09-21T01:11:08.799 INFO:tasks.cephadm:Adding mon.c on smithi098 2024-09-21T01:11:08.799 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c' 2024-09-21T01:11:09.033 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/873218536' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:09.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:09 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:09.692 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mon update... 2024-09-21T01:11:10.275 DEBUG:teuthology.orchestra.run.smithi089:mon.b> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T01:11:10.278 DEBUG:teuthology.orchestra.run.smithi098:mon.c> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T01:11:10.280 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:11:10.280 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:10.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:10 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:11.255 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:11 smithi027 ceph-mon[30375]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:11:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:11 smithi027 ceph-mon[30375]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c;count:3 2024-09-21T01:11:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:11 smithi027 ceph-mon[30375]: Deploying daemon mon.c on smithi098 2024-09-21T01:11:12.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:12 smithi027 ceph-mon[30375]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:12.923 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:11:12.923 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":1,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","modified":"2024-09-21T01:09:30.610244Z","created":"2024-09-21T01:09:30.610244Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T01:11:12.923 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 1 2024-09-21T01:11:13.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:13 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/1371958869' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:11:14.130 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 podman[34353]: 2024-09-21 01:11:14.09872971 +0000 UTC m=+0.315495586 container start ff8ee050bf7aa4aa457158054313b29fc4fa60fa89d18a70d62b909c732cc555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T01:11:14.130 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T01:11:14.130 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T01:11:14.130 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pidfile_write: ignore empty --pid-file 2024-09-21T01:11:14.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: load: jerasure load: lrc 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: RocksDB version: 7.9.2 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Git sha 0 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: DB SUMMARY 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: DB Session ID: IK1QQJNQI4I1IHNEMNGR 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: CURRENT file: CURRENT 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.error_if_exists: 0 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.create_if_missing: 0 2024-09-21T01:11:14.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.paranoid_checks: 1 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.env: 0x5641655cbd60 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.info_log: 0x56416aa15360 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.statistics: (nil) 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.use_fsync: 0 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_log_file_size: 0 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T01:11:14.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.allow_fallocate: 1 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.use_direct_reads: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.db_log_dir: 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.wal_dir: 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T01:11:14.401 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.write_buffer_manager: 0x56416aa19d60 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.unordered_write: 0 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T01:11:14.402 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.row_cache: None 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.wal_filter: None 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.two_write_queues: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.wal_compression: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.atomic_flush: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.log_readahead_size: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T01:11:14.403 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T01:11:14.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T01:11:14.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T01:11:14.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T01:11:14.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_background_jobs: 2 2024-09-21T01:11:14.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_background_compactions: -1 2024-09-21T01:11:14.404 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_subcompactions: 1 2024-09-21T01:11:14.405 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T01:11:14.405 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T01:11:14.405 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T01:11:14.405 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_open_files: -1 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_background_flushes: -1 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Compression algorithms supported: 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kZSTD supported: 0 2024-09-21T01:11:14.406 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kXpressCompression supported: 0 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kBZip2Compression supported: 0 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kLZ4Compression supported: 1 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kZlibCompression supported: 1 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: kSnappyCompression supported: 1 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.merge_operator: 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_filter: None 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T01:11:14.407 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56416aa14fa0) 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: cache_index_and_filter_blocks: 1 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: pin_top_level_index_and_filter: 1 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_type: 0 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: data_block_index_type: 0 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_shortening: 1 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: checksum: 4 2024-09-21T01:11:14.408 INFO:journalctl@ceph.mon.c.smithi098.stdout: no_block_cache: 0 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache: 0x5641669d9610 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_name: BinnedLRUCache 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_options: 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: capacity : 536870912 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: num_shard_bits : 4 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: strict_capacity_limit : 0 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: high_pri_pool_ratio: 0.000 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_cache_compressed: (nil) 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: persistent_cache: (nil) 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_size: 4096 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_size_deviation: 10 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_restart_interval: 16 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: index_block_restart_interval: 1 2024-09-21T01:11:14.409 INFO:journalctl@ceph.mon.c.smithi098.stdout: metadata_block_size: 4096 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: partition_filters: 0 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: use_delta_encoding: 1 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: filter_policy: bloomfilter 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: whole_key_filtering: 1 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: verify_compression: 0 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: read_amp_bytes_per_bit: 0 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: format_version: 5 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: enable_index_compression: 1 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: block_align: 0 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: max_auto_readahead_size: 262144 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: prepopulate_block_cache: 0 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: initial_auto_readahead_size: 8192 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T01:11:14.410 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression: NoCompression 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.num_levels: 7 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T01:11:14.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T01:11:14.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T01:11:14.413 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.inplace_update_support: 0 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T01:11:14.414 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.bloom_locality: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.max_successive_merges: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.ttl: 2592000 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.enable_blob_files: false 2024-09-21T01:11:14.415 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.min_blob_size: 0 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4cb72f19-d941-4bd5-a649-7e1d42ae2359 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726881074134793, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726881074135198, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726881074, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4cb72f19-d941-4bd5-a649-7e1d42ae2359", "db_session_id": "IK1QQJNQI4I1IHNEMNGR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T01:11:14.416 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726881074135289, "job": 1, "event": "recovery_finished"} 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56416aa38a80 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: DB pointer 0x56416b7de000 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** DB Stats ** 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T01:11:14.417 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** Compaction Stats [default] ** 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** Compaction Stats [default] ** 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T01:11:14.418 INFO:journalctl@ceph.mon.c.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: Block cache BinnedLRUCache@0x5641669d9610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T01:11:14.419 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: starting mon.c rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(???) e0 preinit fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: e1 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: btime 2024-09-21T01:09:34:736922+0000 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: legacy client fscid: -1 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout: No filesystems configured 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T01:11:14.420 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Added host smithi089 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:11:14.421 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Deploying cephadm binary to smithi098 2024-09-21T01:11:14.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Added host smithi098 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/873218536' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.423 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/873218536' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:14.424 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Saving service mon spec with placement smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c;count:3 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: Deploying daemon mon.c on smithi098 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/1371958869' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 ceph-mon[34372]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 bash[34353]: ff8ee050bf7aa4aa457158054313b29fc4fa60fa89d18a70d62b909c732cc555 2024-09-21T01:11:14.425 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:14 smithi098 systemd[1]: Started Ceph mon.c for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:11:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:14 smithi027 ceph-mon[30375]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:14.695 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:11:14.695 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:11:14.947 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:11:19.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:19.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: mon.a calling monitor election 2024-09-21T01:11:19.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: mon.c calling monitor election 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: monmap epoch 2 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: last_changed 2024-09-21T01:11:14.166323+0000 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: min_mon_release 19 (squid) 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: election_strategy: 1 2024-09-21T01:11:19.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: fsmap 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: mgrmap e14: x(active, since 46s) 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: overall HEALTH_OK 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:19.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:19 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:19.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:19.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: mon.a calling monitor election 2024-09-21T01:11:19.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: mon.c calling monitor election 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: monmap epoch 2 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: last_changed 2024-09-21T01:11:14.166323+0000 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:19.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: election_strategy: 1 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: fsmap 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: mgrmap e14: x(active, since 46s) 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:19.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:19 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:20.049 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:11:20.049 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","modified":"2024-09-21T01:11:14.166323Z","created":"2024-09-21T01:09:30.610244Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:11:20.049 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 2 2024-09-21T01:11:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:20 smithi098 ceph-mon[34372]: Deploying daemon mon.b on smithi089 2024-09-21T01:11:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:20 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/2870371695' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:11:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:20 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:20.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:20 smithi027 ceph-mon[30375]: Deploying daemon mon.b on smithi089 2024-09-21T01:11:20.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:20 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/2870371695' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:11:20.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:20 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:21 smithi027 ceph-mon[30375]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:21.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:21 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:11:21.166+0000 7f31d6400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T01:11:21.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:21 smithi098 ceph-mon[34372]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:21.705 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:11:21.705 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:11:21.946 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:11:22.642 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:11:22.643 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","modified":"2024-09-21T01:11:14.166323Z","created":"2024-09-21T01:09:30.610244Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:11:22.643 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 2 2024-09-21T01:11:24.244 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T01:11:24.244 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T01:11:24.486 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:11:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: mon.c calling monitor election 2024-09-21T01:11:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: mon.a calling monitor election 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: monmap epoch 3 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:28.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: last_changed 2024-09-21T01:11:22.979431+0000 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: min_mon_release 19 (squid) 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: election_strategy: 1 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: fsmap 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: mgrmap e14: x(active, since 55s) 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: overall HEALTH_OK 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:28 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.411 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:11:28.411 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","modified":"2024-09-21T01:11:22.979431Z","created":"2024-09-21T01:09:30.610244Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T01:11:28.411 INFO:teuthology.orchestra.run.smithi098.stderr:dumped monmap epoch 3 2024-09-21T01:11:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: mon.c calling monitor election 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: mon.a calling monitor election 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T01:11:28.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: monmap epoch 3 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: last_changed 2024-09-21T01:11:22.979431+0000 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: election_strategy: 1 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: fsmap 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: mgrmap e14: x(active, since 55s) 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:28.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:28 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:29.012 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T01:11:29.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T01:11:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:29 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:29 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/2076568191' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:11:29.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:29 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:29.375 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:29 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:29.375 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:29 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/2076568191' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T01:11:29.375 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:29 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:29.407 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:29 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:30.329 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:30.329 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-09-21T01:11:30.329 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:30.329 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-09-21T01:11:30.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: mon.b calling monitor election 2024-09-21T01:11:30.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:11:30.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: mon.b calling monitor election 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: mon.c calling monitor election 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: mon.a calling monitor election 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: monmap epoch 3 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: last_changed 2024-09-21T01:11:22.979431+0000 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: min_mon_release 19 (squid) 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: election_strategy: 1 2024-09-21T01:11:30.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: fsmap 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: mgrmap e14: x(active, since 57s) 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:30.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/874581138' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:30.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:30 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: mon.b calling monitor election 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: mon.b calling monitor election 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: mon.c calling monitor election 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: mon.a calling monitor election 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: monmap epoch 3 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: last_changed 2024-09-21T01:11:22.979431+0000 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: min_mon_release 19 (squid) 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: election_strategy: 1 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: fsmap 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:31.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: mgrmap e14: x(active, since 57s) 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: overall HEALTH_OK 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/874581138' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:31.003 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:31.003 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:30 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:31.128 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T01:11:31.128 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:11:31.129 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:11:31.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: mon.b calling monitor election 2024-09-21T01:11:31.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:11:31.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: mon.b calling monitor election 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: mon.c calling monitor election 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: mon.a calling monitor election 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: monmap epoch 3 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: last_changed 2024-09-21T01:11:22.979431+0000 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: created 2024-09-21T01:09:30.610244+0000 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: min_mon_release 19 (squid) 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: election_strategy: 1 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-09-21T01:11:31.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: 1: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.c 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: fsmap 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: mgrmap e14: x(active, since 57s) 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: overall HEALTH_OK 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/874581138' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:31.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:31.151 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:30 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:31.188 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:11:31.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:31.286 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:11:31.286 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:11:31.319 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:11:31.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:31.388 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:11:31.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T01:11:31.420 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:11:31.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:31.488 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T01:11:31.488 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T01:11:32.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:31 smithi089 ceph-mon[34486]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:32.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:31 smithi089 ceph-mon[34486]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:11:32.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:31 smithi089 ceph-mon[34486]: Reconfiguring daemon mon.a on smithi027 2024-09-21T01:11:32.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:31 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:32.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:31 smithi027 ceph-mon[30375]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:32.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:31 smithi027 ceph-mon[30375]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:11:32.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:31 smithi027 ceph-mon[30375]: Reconfiguring daemon mon.a on smithi027 2024-09-21T01:11:32.083 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:31 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:32.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:31 smithi098 ceph-mon[34372]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:31 smithi098 ceph-mon[34372]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T01:11:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:31 smithi098 ceph-mon[34372]: Reconfiguring daemon mon.a on smithi027 2024-09-21T01:11:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:31 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:32.410 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:33.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:33.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:11:33.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:33.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:33.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:33.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: Reconfiguring daemon mon.b on smithi089 2024-09-21T01:11:33.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4124211637' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T01:11:33.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:33 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T01:11:33.702 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: Reconfiguring daemon mon.b on smithi089 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4124211637' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T01:11:33.703 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:33 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: Reconfiguring mon.b (monmap changed)... 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:33.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: Reconfiguring daemon mon.b on smithi089 2024-09-21T01:11:33.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:33.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4124211637' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T01:11:33.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:33 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T01:11:34.181 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ignoring --setuser ceph since I am not root 2024-09-21T01:11:34.181 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ignoring --setgroup ceph since I am not root 2024-09-21T01:11:34.609 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-09-21T01:11:34.609 INFO:tasks.cephadm:Adding mgr.y on smithi089 2024-09-21T01:11:34.609 INFO:tasks.cephadm:Adding mgr.z on smithi098 2024-09-21T01:11:34.609 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi027=x;smithi089=y;smithi098=z' 2024-09-21T01:11:34.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:34.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:34.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:34.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:34.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:34.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T01:11:34.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:34 smithi098 ceph-mon[34372]: mgrmap e15: x(active, since 61s) 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T01:11:34.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:34 smithi027 ceph-mon[30375]: mgrmap e15: x(active, since 61s) 2024-09-21T01:11:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: from='mgr.14152 172.21.15.27:0/2269400615' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T01:11:35.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:34 smithi089 ceph-mon[34486]: mgrmap e15: x(active, since 61s) 2024-09-21T01:11:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: Active manager daemon x restarted 2024-09-21T01:11:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: Activating manager daemon x 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: mgrmap e16: x(active, starting, since 0.00479031s) 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:11:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:40 smithi098 ceph-mon[34372]: Manager daemon x is now available 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: Active manager daemon x restarted 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: Activating manager daemon x 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: mgrmap e16: x(active, starting, since 0.00479031s) 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:11:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:11:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:40 smithi027 ceph-mon[30375]: Manager daemon x is now available 2024-09-21T01:11:40.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: Active manager daemon x restarted 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: Activating manager daemon x 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: mgrmap e16: x(active, starting, since 0.00479031s) 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T01:11:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:40 smithi089 ceph-mon[34486]: Manager daemon x is now available 2024-09-21T01:11:41.329 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mgr update... 2024-09-21T01:11:41.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:41.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:11:41.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:11:41.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:41 smithi089 ceph-mon[34486]: mgrmap e17: x(active, since 1.01175s) 2024-09-21T01:11:41.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:11:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:11:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:41 smithi098 ceph-mon[34372]: mgrmap e17: x(active, since 1.01175s) 2024-09-21T01:11:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T01:11:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T01:11:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:41 smithi027 ceph-mon[30375]: mgrmap e17: x(active, since 1.01175s) 2024-09-21T01:11:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:41.945 DEBUG:teuthology.orchestra.run.smithi089:mgr.y> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T01:11:41.948 DEBUG:teuthology.orchestra.run.smithi098:mgr.z> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T01:11:41.951 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T01:11:41.951 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T01:11:41.955 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T01:11:41.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T01:11:41.999 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T01:11:42.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T01:11:42.044 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T01:11:42.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T01:11:42.089 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T01:11:42.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T01:11:42.135 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T01:11:42.136 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T01:11:42.188 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T01:11:42.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T01:11:42.237 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T01:11:42.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T01:11:42.293 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T01:11:42.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T01:11:42.353 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T01:11:42.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T01:11:42.405 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T01:11:42.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T01:11:42.458 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T01:11:42.458 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T01:11:42.521 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T01:11:42.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T01:11:42.579 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T01:11:42.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T01:11:42.631 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T01:11:42.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T01:11:42.684 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T01:11:42.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T01:11:42.734 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T01:11:42.734 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T01:11:42.786 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T01:11:42.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:11:42.703+0000 7f6551a00640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:01:11:42] ENGINE Error in HTTPServer.serve 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: Traceback (most recent call last): 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._connections.run(self.expiration_interval) 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._run(expiration_interval) 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:11:42.795 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: s = self.context.wrap_socket( 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: return self.sslsocket_class._create( 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self.do_handshake() 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: self._sslobj.do_handshake() 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:11:42.796 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:11:42 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:11:42.805 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T01:11:42.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T01:11:42.859 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T01:11:42.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T01:11:42.921 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T01:11:42.922 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T01:11:43.009 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T01:11:43.009 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: Saving service mgr spec with placement smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.045 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:43.072 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T01:11:43.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T01:11:43.128 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T01:11:43.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T01:11:43.185 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T01:11:43.185 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T01:11:43.251 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T01:11:43.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T01:11:43.315 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T01:11:43.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T01:11:43.316 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: Saving service mgr spec with placement smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T01:11:43.316 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.317 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:43.369 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T01:11:43.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T01:11:43.425 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T01:11:43.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T01:11:43.480 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T01:11:43.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: Saving service mgr spec with placement smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:43.523 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.535 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T01:11:43.535 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi027') 2024-09-21T01:11:43.742 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.813 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:43.847 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: [21/Sep/2024:01:11:42] ENGINE Bus STARTING 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: [21/Sep/2024:01:11:42] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: [21/Sep/2024:01:11:42] ENGINE Error in HTTPServer.serve 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout: Traceback (most recent call last): 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:11:44.251 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._run(expiration_interval) 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: s = self.context.wrap_socket( 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: return self.sslsocket_class._create( 2024-09-21T01:11:44.252 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout: self.do_handshake() 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout: self._sslobj.do_handshake() 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: [21/Sep/2024:01:11:42] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: [21/Sep/2024:01:11:42] ENGINE Bus STARTED 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: mgrmap e18: x(active, since 2s) 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.253 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:44 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: [21/Sep/2024:01:11:42] ENGINE Bus STARTING 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: [21/Sep/2024:01:11:42] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: [21/Sep/2024:01:11:42] ENGINE Error in HTTPServer.serve 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout: Traceback (most recent call last): 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:11:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: self._run(expiration_interval) 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: s = self.context.wrap_socket( 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: return self.sslsocket_class._create( 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: self.do_handshake() 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: self._sslobj.do_handshake() 2024-09-21T01:11:44.399 INFO:journalctl@ceph.mon.c.smithi098.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: [21/Sep/2024:01:11:42] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: [21/Sep/2024:01:11:42] ENGINE Bus STARTED 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: mgrmap e18: x(active, since 2s) 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:44 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: [21/Sep/2024:01:11:42] ENGINE Bus STARTING 2024-09-21T01:11:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: [21/Sep/2024:01:11:42] ENGINE Serving on https://172.21.15.27:7150 2024-09-21T01:11:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: [21/Sep/2024:01:11:42] ENGINE Error in HTTPServer.serve 2024-09-21T01:11:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout: Traceback (most recent call last): 2024-09-21T01:11:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: self._connections.run(self.expiration_interval) 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: self._run(expiration_interval) 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: s = self.context.wrap_socket( 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: return self.sslsocket_class._create( 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: self.do_handshake() 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T01:11:44.501 INFO:journalctl@ceph.mon.b.smithi089.stdout: self._sslobj.do_handshake() 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: [21/Sep/2024:01:11:42] ENGINE Serving on http://172.21.15.27:8765 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: [21/Sep/2024:01:11:42] ENGINE Bus STARTED 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: mgrmap e18: x(active, since 2s) 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:44.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:44 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:46.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:46 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3180898467' entity='client.admin' 2024-09-21T01:11:46.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:46 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:46.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:46 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1321685982' entity='client.admin' 2024-09-21T01:11:46.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:46 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3180898467' entity='client.admin' 2024-09-21T01:11:46.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:46 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:46.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:46 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1321685982' entity='client.admin' 2024-09-21T01:11:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:46 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3180898467' entity='client.admin' 2024-09-21T01:11:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:46 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:46 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1321685982' entity='client.admin' 2024-09-21T01:11:47.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:47 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3004629101' entity='client.admin' 2024-09-21T01:11:47.633 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:47.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:47 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3004629101' entity='client.admin' 2024-09-21T01:11:47.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:47 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3004629101' entity='client.admin' 2024-09-21T01:11:49.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1648301434' entity='client.admin' 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:49.323 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:49 smithi027 ceph-mon[30375]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1648301434' entity='client.admin' 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:49.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:49.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:49 smithi098 ceph-mon[34372]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1648301434' entity='client.admin' 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"} : dispatch 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi027:/etc/ceph/ceph.conf 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi089:/etc/ceph/ceph.conf 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi098:/etc/ceph/ceph.conf 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:49.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T01:11:49.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:49.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:49 smithi089 ceph-mon[34486]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:50.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 -- 172.21.15.27:0/4169082409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc54003fb70 msgr2=0x7fc540042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 --2- 172.21.15.27:0/4169082409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc54003fb70 0x7fc540042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009c90 tx=0x7fc54c005f90 comp rx=0 tx=0).stop 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 -- 172.21.15.27:0/4169082409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc55c1047f0 msgr2=0x7fc55c19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 --2- 172.21.15.27:0/4169082409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc55c1047f0 0x7fc55c19c7b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc54802aa70 tx=0x7fc548003c10 comp rx=0 tx=0).stop 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 -- 172.21.15.27:0/4169082409 shutdown_connections 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 --2- 172.21.15.27:0/4169082409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc54003fb70 0x7fc540042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 --2- 172.21.15.27:0/4169082409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc55c105190 0x7fc55c19d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 --2- 172.21.15.27:0/4169082409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc55c1047f0 0x7fc55c19c7b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 --2- 172.21.15.27:0/4169082409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc55c103760 0x7fc55c19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.029+0000 7fc562200640 1 -- 172.21.15.27:0/4169082409 >> 172.21.15.27:0/4169082409 conn(0x7fc55c0ff0b0 msgr2=0x7fc55c0755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.030+0000 7fc562200640 1 -- 172.21.15.27:0/4169082409 shutdown_connections 2024-09-21T01:11:50.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:50.030+0000 7fc562200640 1 -- 172.21.15.27:0/4169082409 wait complete. 2024-09-21T01:11:50.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.496 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: Deploying daemon mgr.z on smithi098 2024-09-21T01:11:50.497 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:50 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: Deploying daemon mgr.z on smithi098 2024-09-21T01:11:50.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:50 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: Updating smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: Updating smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: Updating smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T01:11:50.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: Deploying daemon mgr.z on smithi098 2024-09-21T01:11:50.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:50 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 -- 172.21.15.27:0/3343190685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 msgr2=0x7f4aa810b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 --2- 172.21.15.27:0/3343190685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa810b910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009900 tx=0x7f4a9002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 -- 172.21.15.27:0/3343190685 shutdown_connections 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 --2- 172.21.15.27:0/3343190685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa810be50 0x7f4aa8112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 --2- 172.21.15.27:0/3343190685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa810b910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 --2- 172.21.15.27:0/3343190685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4aa8105bc0 0x7f4aa8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 -- 172.21.15.27:0/3343190685 >> 172.21.15.27:0/3343190685 conn(0x7f4aa8101670 msgr2=0x7f4aa8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 -- 172.21.15.27:0/3343190685 shutdown_connections 2024-09-21T01:11:51.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.243+0000 7f4aaca00640 1 -- 172.21.15.27:0/3343190685 wait complete. 2024-09-21T01:11:51.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.244+0000 7f4aaca00640 1 Processor -- start 2024-09-21T01:11:51.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 -- start start 2024-09-21T01:11:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 0x7f4aa819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4aa810be50 0x7f4aa81a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8115b60 con 0x7f4aa810b500 2024-09-21T01:11:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa81159e0 con 0x7f4aa8105bc0 2024-09-21T01:11:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aaca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa8115ce0 con 0x7f4aa810be50 2024-09-21T01:11:51.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4a9ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:51.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aa7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 0x7f4aa819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:51.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4a9ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51698/0 (socket says 172.21.15.27:51698) 2024-09-21T01:11:51.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4a9ea00640 1 -- 172.21.15.27:0/2421094000 learned_addr learned my addr 172.21.15.27:0/2421094000 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:11:51.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aa7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 0x7f4aa819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58880/0 (socket says 172.21.15.27:58880) 2024-09-21T01:11:51.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.245+0000 7f4aa7e00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4aa810be50 0x7f4aa81a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa7400640 1 -- 172.21.15.27:0/2421094000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4aa810be50 msgr2=0x7f4aa81a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa7400640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4aa810be50 0x7f4aa81a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa7400640 1 -- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 msgr2=0x7f4aa819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa7400640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa7400640 1 -- 172.21.15.27:0/2421094000 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa81a24c0 con 0x7f4aa8105bc0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa7400640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 0x7f4aa819c550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800ec80 tx=0x7f4a9800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003b70 con 0x7f4aa8105bc0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a98003d10 con 0x7f4aa8105bc0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98010870 con 0x7f4aa8105bc0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.246+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa81a27b0 con 0x7f4aa8105bc0 2024-09-21T01:11:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.247+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa81a8f10 con 0x7f4aa8105bc0 2024-09-21T01:11:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.248+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa8105fd0 con 0x7f4aa8105bc0 2024-09-21T01:11:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.248+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f4a98004020 con 0x7f4aa8105bc0 2024-09-21T01:11:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.248+0000 7f4aa4c00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4a8c03fb70 0x7f4a8c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.248+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4a980549b0 con 0x7f4aa8105bc0 2024-09-21T01:11:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.249+0000 7f4a9ea00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4a8c03fb70 0x7f4a8c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.249+0000 7f4a9ea00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4a8c03fb70 0x7f4a8c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4a90006010 tx=0x7f4a90005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.254+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4a98013190 con 0x7f4aa8105bc0 2024-09-21T01:11:51.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.456+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f4aa802e640 con 0x7f4aa8105bc0 2024-09-21T01:11:51.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.462+0000 7f4aa4c00640 1 -- 172.21.15.27:0/2421094000 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v18)=0 v18) ==== 116+0+0 (secure 0 0 0) 0x7f4a98018dd0 con 0x7f4aa8105bc0 2024-09-21T01:11:51.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4a8c03fb70 msgr2=0x7f4a8c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:51.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4a8c03fb70 0x7f4a8c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4a90006010 tx=0x7f4a90005a20 comp rx=0 tx=0).stop 2024-09-21T01:11:51.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 msgr2=0x7f4aa819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 0x7f4aa819c550 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800ec80 tx=0x7f4a9800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 shutdown_connections 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4a8c03fb70 0x7f4a8c042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4aa810be50 0x7f4aa81a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4aa810b500 0x7f4aa819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 --2- 172.21.15.27:0/2421094000 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4aa8105bc0 0x7f4aa819c550 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.464+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 >> 172.21.15.27:0/2421094000 conn(0x7f4aa8101670 msgr2=0x7f4aa8103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.465+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 shutdown_connections 2024-09-21T01:11:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:51.465+0000 7f4aaca00640 1 -- 172.21.15.27:0/2421094000 wait complete. 2024-09-21T01:11:51.830 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:51 smithi098 systemd[1]: Starting Ceph mgr.z for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:11:52.106 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:51 smithi098 podman[36327]: 2024-09-21 01:11:51.732328739 +0000 UTC m=+0.015694795 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:11:52.107 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:51 smithi098 podman[36327]: 2024-09-21 01:11:51.831960597 +0000 UTC m=+0.115326652 container create 3a4296d6352babfb23c3f0e27a3fcd146a5f4c2be2019a6b6db699fb2bcc3c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T01:11:52.107 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:52 smithi098 podman[36327]: 2024-09-21 01:11:52.067982319 +0000 UTC m=+0.351348398 container init 3a4296d6352babfb23c3f0e27a3fcd146a5f4c2be2019a6b6db699fb2bcc3c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-z, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T01:11:52.107 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:52 smithi098 podman[36327]: 2024-09-21 01:11:52.071499274 +0000 UTC m=+0.354865442 container start 3a4296d6352babfb23c3f0e27a3fcd146a5f4c2be2019a6b6db699fb2bcc3c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T01:11:52.107 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mgr[36346]: -- 172.21.15.98:0/4054260866 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d872830e0 con 0x562d872a1a80 2024-09-21T01:11:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:52.362 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:52.398 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:52 smithi098 bash[36327]: 3a4296d6352babfb23c3f0e27a3fcd146a5f4c2be2019a6b6db699fb2bcc3c20 2024-09-21T01:11:52.398 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:11:52 smithi098 systemd[1]: Started Ceph mgr.z for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:52.491 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:11:52.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:52.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:53.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:53 smithi098 ceph-mon[34372]: Deploying daemon mgr.y on smithi089 2024-09-21T01:11:53.677 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:53 smithi089 ceph-mon[34486]: Deploying daemon mgr.y on smithi089 2024-09-21T01:11:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:53 smithi027 ceph-mon[30375]: Deploying daemon mgr.y on smithi089 2024-09-21T01:11:54.750 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:11:54 smithi089 systemd[1]: Started Ceph mgr.y for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:11:54.784 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi027') 2024-09-21T01:11:54.784 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.434 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:55.435 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:55.437 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:55.437 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:55.529 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.529 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.529 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.530 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.530 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.530 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi027') 2024-09-21T01:11:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.677 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:55.934 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:55.935 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:11:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.388+0000 7f9b09400640 1 -- 172.21.15.27:0/2419380381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 msgr2=0x7f9b04109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.388+0000 7f9b09400640 1 --2- 172.21.15.27:0/2419380381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04109700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009860 tx=0x7f9af002a340 comp rx=0 tx=0).stop 2024-09-21T01:11:56.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.388+0000 7f9b09400640 1 -- 172.21.15.27:0/2419380381 shutdown_connections 2024-09-21T01:11:56.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.388+0000 7f9b09400640 1 --2- 172.21.15.27:0/2419380381 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b04109c40 0x7f9b04110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.389+0000 7f9b09400640 1 --2- 172.21.15.27:0/2419380381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04109700 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.389+0000 7f9b09400640 1 --2- 172.21.15.27:0/2419380381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b041039b0 0x7f9b04103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.389+0000 7f9b09400640 1 -- 172.21.15.27:0/2419380381 >> 172.21.15.27:0/2419380381 conn(0x7f9b040ff440 msgr2=0x7f9b04101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:56.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.389+0000 7f9b09400640 1 -- 172.21.15.27:0/2419380381 shutdown_connections 2024-09-21T01:11:56.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.389+0000 7f9b09400640 1 -- 172.21.15.27:0/2419380381 wait complete. 2024-09-21T01:11:56.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.390+0000 7f9b09400640 1 Processor -- start 2024-09-21T01:11:56.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.390+0000 7f9b09400640 1 -- start start 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b09400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041039b0 0x7f9b04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b09400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b09400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b04109c40 0x7f9b04076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b09400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04113900 con 0x7f9b04109c40 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b09400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04113780 con 0x7f9b041039b0 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b09400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04113a80 con 0x7f9b041092f0 2024-09-21T01:11:56.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b03400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b03400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39788/0 (socket says 172.21.15.27:39788) 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b03400640 1 -- 172.21.15.27:0/3836772895 learned_addr learned my addr 172.21.15.27:0/3836772895 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b08a00640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b04109c40 0x7f9b04076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.391+0000 7f9b03e00640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041039b0 0x7f9b04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03400640 1 -- 172.21.15.27:0/3836772895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041039b0 msgr2=0x7f9b04075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041039b0 0x7f9b04075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03400640 1 -- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b04109c40 msgr2=0x7f9b04076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b04109c40 0x7f9b04076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03400640 1 -- 172.21.15.27:0/3836772895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b041a8590 con 0x7f9b041092f0 2024-09-21T01:11:56.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03e00640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041039b0 0x7f9b04075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:11:56.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b03400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04075c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9af0035040 tx=0x7f9af0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:56.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0037040 con 0x7f9b041092f0 2024-09-21T01:11:56.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af0030c10 con 0x7f9b041092f0 2024-09-21T01:11:56.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.392+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b041a8820 con 0x7f9b041092f0 2024-09-21T01:11:56.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.393+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00035f0 con 0x7f9b041092f0 2024-09-21T01:11:56.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.393+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041a8cf0 con 0x7f9b041092f0 2024-09-21T01:11:56.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.394+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b04103dc0 con 0x7f9b041092f0 2024-09-21T01:11:56.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.395+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f9af0043030 con 0x7f9b041092f0 2024-09-21T01:11:56.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.395+0000 7f9b00c00640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9ae803fb70 0x7f9ae8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.395+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9af0073c60 con 0x7f9b041092f0 2024-09-21T01:11:56.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.395+0000 7f9b03e00640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9ae803fb70 0x7f9ae8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.396+0000 7f9b03e00640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9ae803fb70 0x7f9ae8042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b04077160 tx=0x7f9af4009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:56.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.401+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9af004a060 con 0x7f9b041092f0 2024-09-21T01:11:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 -- 172.21.15.27:0/1161822263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 msgr2=0x7f94fc103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 --2- 172.21.15.27:0/1161822263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc103dc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009900 tx=0x7f94ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 -- 172.21.15.27:0/1161822263 shutdown_connections 2024-09-21T01:11:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 --2- 172.21.15.27:0/1161822263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94fc109c40 0x7f94fc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 --2- 172.21.15.27:0/1161822263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc1092f0 0x7f94fc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 --2- 172.21.15.27:0/1161822263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc103dc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 -- 172.21.15.27:0/1161822263 >> 172.21.15.27:0/1161822263 conn(0x7f94fc0ff440 msgr2=0x7f94fc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 -- 172.21.15.27:0/1161822263 shutdown_connections 2024-09-21T01:11:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.269+0000 7f94fb200640 1 -- 172.21.15.27:0/1161822263 wait complete. 2024-09-21T01:11:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.270+0000 7f94fb200640 1 Processor -- start 2024-09-21T01:11:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.270+0000 7f94fb200640 1 -- start start 2024-09-21T01:11:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc1092f0 0x7f94fc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94fc109c40 0x7f94fc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc113900 con 0x7f94fc109c40 2024-09-21T01:11:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc113780 con 0x7f94fc1092f0 2024-09-21T01:11:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc113a80 con 0x7f94fc1039b0 2024-09-21T01:11:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc1092f0 0x7f94fc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39778/0 (socket says 172.21.15.27:39778) 2024-09-21T01:11:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 -- 172.21.15.27:0/3841854961 learned_addr learned my addr 172.21.15.27:0/3841854961 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:11:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94fa800640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94fc109c40 0x7f94fc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 -- 172.21.15.27:0/3841854961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc1092f0 msgr2=0x7f94fc19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc1092f0 0x7f94fc19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 -- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94fc109c40 msgr2=0x7f94fc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.271+0000 7f94f9e00640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94fc109c40 0x7f94fc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94f9e00640 1 -- 172.21.15.27:0/3841854961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc1a0280 con 0x7f94fc1039b0 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94f9e00640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc19a310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005940 tx=0x7f94ec0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec03f030 con 0x7f94fc1039b0 2024-09-21T01:11:56.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94ec0044c0 con 0x7f94fc1039b0 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc1a0510 con 0x7f94fc1039b0 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec035070 con 0x7f94fc1039b0 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.272+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc1a6cd0 con 0x7f94fc1039b0 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.274+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc103dc0 con 0x7f94fc1039b0 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.274+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7f94ec042030 con 0x7f94fc1039b0 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.275+0000 7f94f2a00640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f94e003fb70 0x7f94e0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.275+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f94ec074aa0 con 0x7f94fc1039b0 2024-09-21T01:11:56.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.275+0000 7f94f9400640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f94e003fb70 0x7f94e0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.276+0000 7f94f9400640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f94e003fb70 0x7f94e0042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f94fc108e70 tx=0x7f94e8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:56.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.280+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94ec038350 con 0x7f94fc1039b0 2024-09-21T01:11:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.498+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f94fc10ba70 con 0x7f94fc1039b0 2024-09-21T01:11:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.504+0000 7f94f2a00640 1 -- 172.21.15.27:0/3841854961 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v19)=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f94ec045800 con 0x7f94fc1039b0 2024-09-21T01:11:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f94e003fb70 msgr2=0x7f94e0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f94e003fb70 0x7f94e0042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f94fc108e70 tx=0x7f94e8005f90 comp rx=0 tx=0).stop 2024-09-21T01:11:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 msgr2=0x7f94fc19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc19a310 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005940 tx=0x7f94ec0040c0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 shutdown_connections 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f94e003fb70 0x7f94e0042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94fc109c40 0x7f94fc19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f94fc1092f0 0x7f94fc19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 --2- 172.21.15.27:0/3841854961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94fc1039b0 0x7f94fc19a310 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.507+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 >> 172.21.15.27:0/3841854961 conn(0x7f94fc0ff440 msgr2=0x7f94fc101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.508+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 shutdown_connections 2024-09-21T01:11:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.508+0000 7f94fb200640 1 -- 172.21.15.27:0/3841854961 wait complete. 2024-09-21T01:11:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.632+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f9b04076a10 con 0x7f9b041092f0 2024-09-21T01:11:56.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.637+0000 7f9b00c00640 1 -- 172.21.15.27:0/3836772895 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v20)=0 v20) ==== 116+0+0 (secure 0 0 0) 0x7f9af0030dc0 con 0x7f9b041092f0 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9ae803fb70 msgr2=0x7f9ae8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9ae803fb70 0x7f9ae8042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b04077160 tx=0x7f9af4009260 comp rx=0 tx=0).stop 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 msgr2=0x7f9b04075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04075c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9af0035040 tx=0x7f9af0032d50 comp rx=0 tx=0).stop 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 shutdown_connections 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9ae803fb70 0x7f9ae8042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b04109c40 0x7f9b04076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b041092f0 0x7f9b04075c50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 --2- 172.21.15.27:0/3836772895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b041039b0 0x7f9b04075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 >> 172.21.15.27:0/3836772895 conn(0x7f9b040ff440 msgr2=0x7f9b04101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 shutdown_connections 2024-09-21T01:11:56.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.640+0000 7f9b09400640 1 -- 172.21.15.27:0/3836772895 wait complete. 2024-09-21T01:11:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 -- 172.21.15.27:0/562431376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49010b6d0 msgr2=0x7fb49010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 --2- 172.21.15.27:0/562431376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49010b6d0 0x7fb49010bae0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb478009900 tx=0x7fb47802a400 comp rx=0 tx=0).stop 2024-09-21T01:11:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 -- 172.21.15.27:0/562431376 shutdown_connections 2024-09-21T01:11:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 --2- 172.21.15.27:0/562431376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010c020 0x7fb490112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 --2- 172.21.15.27:0/562431376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb49010b6d0 0x7fb49010bae0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 --2- 172.21.15.27:0/562431376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb490105d90 0x7fb4901061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 -- 172.21.15.27:0/562431376 >> 172.21.15.27:0/562431376 conn(0x7fb490101860 msgr2=0x7fb490103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 -- 172.21.15.27:0/562431376 shutdown_connections 2024-09-21T01:11:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.722+0000 7fb497400640 1 -- 172.21.15.27:0/562431376 wait complete. 2024-09-21T01:11:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.723+0000 7fb497400640 1 Processor -- start 2024-09-21T01:11:56.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 -- start start 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb490105d90 0x7fb49019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 0x7fb49019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb49010c020 0x7fb4901a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb490115d30 con 0x7fb490105d90 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb490115bb0 con 0x7fb49010c020 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb497400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb490115eb0 con 0x7fb49010b6d0 2024-09-21T01:11:56.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb494a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb490105d90 0x7fb49019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb487e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 0x7fb49019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb487e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 0x7fb49019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39798/0 (socket says 172.21.15.27:39798) 2024-09-21T01:11:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb487e00640 1 -- 172.21.15.27:0/1210891383 learned_addr learned my addr 172.21.15.27:0/1210891383 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:11:56.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.724+0000 7fb495400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb49010c020 0x7fb4901a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb487e00640 1 -- 172.21.15.27:0/1210891383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb49010c020 msgr2=0x7fb4901a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb487e00640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb49010c020 0x7fb4901a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb487e00640 1 -- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb490105d90 msgr2=0x7fb49019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb487e00640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb490105d90 0x7fb49019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb487e00640 1 -- 172.21.15.27:0/1210891383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4901a2730 con 0x7fb49010b6d0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb487e00640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 0x7fb49019cd60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb478005ab0 tx=0x7fb478003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47803f070 con 0x7fb49010b6d0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478031b50 con 0x7fb49010b6d0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4901a29c0 con 0x7fb49010b6d0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478035030 con 0x7fb49010b6d0 2024-09-21T01:11:56.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.725+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4901a9180 con 0x7fb49010b6d0 2024-09-21T01:11:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.726+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb490107010 con 0x7fb49010b6d0 2024-09-21T01:11:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.727+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 51915+0+0 (secure 0 0 0) 0x7fb478042070 con 0x7fb49010b6d0 2024-09-21T01:11:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.727+0000 7fb48e000640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb47403fb70 0x7fb474042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.727+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb478074cf0 con 0x7fb49010b6d0 2024-09-21T01:11:56.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.727+0000 7fb494a00640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb47403fb70 0x7fb474042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:56.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.728+0000 7fb494a00640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb47403fb70 0x7fb474042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb480005fd0 tx=0x7fb480009500 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:56.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.732+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb47803b460 con 0x7fb49010b6d0 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:56.878 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:56.879 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:56.879 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T01:11:56.879 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:56.879 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:56 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:56.890 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:56.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.979+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fb4901061a0 con 0x7fb49010b6d0 2024-09-21T01:11:56.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.986+0000 7fb48e000640 1 -- 172.21.15.27:0/1210891383 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fb478031d00 con 0x7fb49010b6d0 2024-09-21T01:11:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb47403fb70 msgr2=0x7fb474042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb47403fb70 0x7fb474042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb480005fd0 tx=0x7fb480009500 comp rx=0 tx=0).stop 2024-09-21T01:11:56.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 msgr2=0x7fb49019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 0x7fb49019cd60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb478005ab0 tx=0x7fb478003b90 comp rx=0 tx=0).stop 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 shutdown_connections 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb47403fb70 0x7fb474042080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb49010c020 0x7fb4901a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb49010b6d0 0x7fb49019cd60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 --2- 172.21.15.27:0/1210891383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb490105d90 0x7fb49019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 >> 172.21.15.27:0/1210891383 conn(0x7fb490101860 msgr2=0x7fb490108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 shutdown_connections 2024-09-21T01:11:56.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:56.989+0000 7fb497400640 1 -- 172.21.15.27:0/1210891383 wait complete. 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:57.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T01:11:57.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:57.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:56 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T01:11:57.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:11:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: Reconfiguring daemon mgr.x on smithi027 2024-09-21T01:11:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:56 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:57 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:11:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:57 smithi027 ceph-mon[30375]: Standby manager daemon z started 2024-09-21T01:11:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:57 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:11:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:57 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:11:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:57 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:11:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:57 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:11:58.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:57 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:11:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:57 smithi089 ceph-mon[34486]: Standby manager daemon z started 2024-09-21T01:11:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:57 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:11:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:57 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:11:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:57 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:11:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:57 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:11:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:57 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:11:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:57 smithi098 ceph-mon[34372]: Standby manager daemon z started 2024-09-21T01:11:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:57 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T01:11:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:57 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:11:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:57 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T01:11:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:57 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.98:0/1299197317' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:11:59.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:58 smithi027 ceph-mon[30375]: mgrmap e19: x(active, since 17s), standbys: z 2024-09-21T01:11:59.027 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:11:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 -- 172.21.15.27:0/4278641992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 msgr2=0x7f7bbc1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 --2- 172.21.15.27:0/4278641992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc1027f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009900 tx=0x7f7bb002a480 comp rx=0 tx=0).stop 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 -- 172.21.15.27:0/4278641992 shutdown_connections 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 --2- 172.21.15.27:0/4278641992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bbc10ff90 0x7f7bbc1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 --2- 172.21.15.27:0/4278641992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 0x7f7bbc10fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 --2- 172.21.15.27:0/4278641992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc1027f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.026+0000 7f7bc2800640 1 -- 172.21.15.27:0/4278641992 >> 172.21.15.27:0/4278641992 conn(0x7f7bbc0fe2c0 msgr2=0x7f7bbc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.027+0000 7f7bc2800640 1 -- 172.21.15.27:0/4278641992 shutdown_connections 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.027+0000 7f7bc2800640 1 -- 172.21.15.27:0/4278641992 wait complete. 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 Processor -- start 2024-09-21T01:11:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 -- start start 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 0x7f7bbc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bbc10ff90 0x7f7bbc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc115a80 con 0x7f7bbc102d30 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc115900 con 0x7f7bbc1023e0 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.028+0000 7f7bc2800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc115c00 con 0x7f7bbc10ff90 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53958/0 (socket says 172.21.15.27:53958) 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 -- 172.21.15.27:0/3507101782 learned_addr learned my addr 172.21.15.27:0/3507101782 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc0a00640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 0x7f7bbc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1e00640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bbc10ff90 0x7f7bbc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 -- 172.21.15.27:0/3507101782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bbc10ff90 msgr2=0x7f7bbc1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bbc10ff90 0x7f7bbc1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 -- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 msgr2=0x7f7bbc19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 0x7f7bbc19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 -- 172.21.15.27:0/3507101782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bbc1a23c0 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc0a00640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 0x7f7bbc19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.029+0000 7f7bc1400640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc19c450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005940 tx=0x7f7bb00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.030+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb003f030 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.030+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb00044c0 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.030+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0035070 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.030+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bbc1a2650 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.030+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bbc1a8e10 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.031+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bbc1027f0 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.032+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f7bb0042030 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.033+0000 7f7bba000640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ba407b020 0x7f7ba407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.033+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7bb00bcee0 con 0x7f7bbc1023e0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.033+0000 7f7bc0a00640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ba407b020 0x7f7ba407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.034+0000 7f7bc0a00640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ba407b020 0x7f7ba407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc111270 tx=0x7f7ba8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:59.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.037+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7bb0089850 con 0x7f7bbc1023e0 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 -- 172.21.15.27:0/2069790847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181092f0 msgr2=0x7fba18109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 --2- 172.21.15.27:0/2069790847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181092f0 0x7fba18109700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fba04009900 tx=0x7fba0402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 -- 172.21.15.27:0/2069790847 shutdown_connections 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 --2- 172.21.15.27:0/2069790847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba18109c40 0x7fba18110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 --2- 172.21.15.27:0/2069790847 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181092f0 0x7fba18109700 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 --2- 172.21.15.27:0/2069790847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba181039b0 0x7fba18103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 -- 172.21.15.27:0/2069790847 >> 172.21.15.27:0/2069790847 conn(0x7fba180ff440 msgr2=0x7fba18101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 -- 172.21.15.27:0/2069790847 shutdown_connections 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.117+0000 7fba1ce00640 1 -- 172.21.15.27:0/2069790847 wait complete. 2024-09-21T01:11:59.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 Processor -- start 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 -- start start 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 0x7fba181980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba181092f0 0x7fba18198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba18109c40 0x7fba1819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba18111740 con 0x7fba18109c40 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba181115c0 con 0x7fba181039b0 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba1ce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fba181118c0 con 0x7fba181092f0 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba17400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 0x7fba181980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba17400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 0x7fba181980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53972/0 (socket says 172.21.15.27:53972) 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.119+0000 7fba17400640 1 -- 172.21.15.27:0/1051592442 learned_addr learned my addr 172.21.15.27:0/1051592442 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17e00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba18109c40 0x7fba1819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba16a00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba181092f0 0x7fba18198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17400640 1 -- 172.21.15.27:0/1051592442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba181092f0 msgr2=0x7fba18198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17400640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba181092f0 0x7fba18198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17400640 1 -- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba18109c40 msgr2=0x7fba1819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17400640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba18109c40 0x7fba1819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17400640 1 -- 172.21.15.27:0/1051592442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1819e060 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba17400640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 0x7fba181980f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e9f0 tx=0x7fba0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.120+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0800cd80 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.121+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba080108e0 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.121+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1819e350 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.121+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08018980 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.121+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba181a4ab0 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.122+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba18103dc0 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.123+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7fba08010430 con 0x7fba181039b0 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.124+0000 7fba0fe00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb9fc07b020 0x7fb9fc07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:11:59.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.124+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fba0809d210 con 0x7fba181039b0 2024-09-21T01:11:59.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.124+0000 7fba16a00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb9fc07b020 0x7fb9fc07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:11:59.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.125+0000 7fba16a00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb9fc07b020 0x7fb9fc07d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fba0402a9e0 tx=0x7fba04005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:11:59.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.128+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba08069c40 con 0x7fba181039b0 2024-09-21T01:11:59.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:58 smithi089 ceph-mon[34486]: mgrmap e19: x(active, since 17s), standbys: z 2024-09-21T01:11:59.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:11:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:11:59.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.251+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f7bbc10b120 con 0x7f7bbc1023e0 2024-09-21T01:11:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.257+0000 7f7bba000640 1 -- 172.21.15.27:0/3507101782 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v22)=0 v22) ==== 111+0+0 (secure 0 0 0) 0x7f7bb008e700 con 0x7f7bbc1023e0 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ba407b020 msgr2=0x7f7ba407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ba407b020 0x7f7ba407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc111270 tx=0x7f7ba8005f90 comp rx=0 tx=0).stop 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 msgr2=0x7f7bbc19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc19c450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005940 tx=0x7f7bb00040c0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 shutdown_connections 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ba407b020 0x7f7ba407d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bbc10ff90 0x7f7bbc1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bbc102d30 0x7f7bbc19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 --2- 172.21.15.27:0/3507101782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7bbc1023e0 0x7f7bbc19c450 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 >> 172.21.15.27:0/3507101782 conn(0x7f7bbc0fe2c0 msgr2=0x7f7bbc112130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 shutdown_connections 2024-09-21T01:11:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.260+0000 7f7bc2800640 1 -- 172.21.15.27:0/3507101782 wait complete. 2024-09-21T01:11:59.303 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:59.308 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:59.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.368+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fba1802e640 con 0x7fba181039b0 2024-09-21T01:11:59.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.374+0000 7fba0fe00640 1 -- 172.21.15.27:0/1051592442 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v23)=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fba0806eaf0 con 0x7fba181039b0 2024-09-21T01:11:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb9fc07b020 msgr2=0x7fb9fc07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb9fc07b020 0x7fb9fc07d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fba0402a9e0 tx=0x7fba04005a20 comp rx=0 tx=0).stop 2024-09-21T01:11:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 msgr2=0x7fba181980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:11:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 0x7fba181980f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fba0800e9f0 tx=0x7fba0800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 shutdown_connections 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb9fc07b020 0x7fb9fc07d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba18109c40 0x7fba1819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba181092f0 0x7fba18198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 --2- 172.21.15.27:0/1051592442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba181039b0 0x7fba181980f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 >> 172.21.15.27:0/1051592442 conn(0x7fba180ff440 msgr2=0x7fba18101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.377+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 shutdown_connections 2024-09-21T01:11:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.378+0000 7fba1ce00640 1 -- 172.21.15.27:0/1051592442 wait complete. 2024-09-21T01:11:59.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:58 smithi098 ceph-mon[34372]: mgrmap e19: x(active, since 17s), standbys: z 2024-09-21T01:11:59.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:11:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T01:11:59.939 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:59.939 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:11:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:12:00.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.357+0000 7f78de600640 1 -- 172.21.15.27:0/787367948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d8105d80 msgr2=0x7f78d8106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.357+0000 7f78de600640 1 --2- 172.21.15.27:0/787367948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d8105d80 0x7f78d8106190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009900 tx=0x7f78c802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.357+0000 7f78de600640 1 -- 172.21.15.27:0/787367948 shutdown_connections 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.357+0000 7f78de600640 1 --2- 172.21.15.27:0/787367948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78d810c010 0x7f78d8112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.357+0000 7f78de600640 1 --2- 172.21.15.27:0/787367948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810b6c0 0x7f78d810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.357+0000 7f78de600640 1 --2- 172.21.15.27:0/787367948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d8105d80 0x7f78d8106190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.358+0000 7f78de600640 1 -- 172.21.15.27:0/787367948 >> 172.21.15.27:0/787367948 conn(0x7f78d81017f0 msgr2=0x7f78d8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.358+0000 7f78de600640 1 -- 172.21.15.27:0/787367948 shutdown_connections 2024-09-21T01:12:00.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.358+0000 7f78de600640 1 -- 172.21.15.27:0/787367948 wait complete. 2024-09-21T01:12:00.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.359+0000 7f78de600640 1 Processor -- start 2024-09-21T01:12:00.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.359+0000 7f78de600640 1 -- start start 2024-09-21T01:12:00.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78de600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78d8105d80 0x7f78d819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:00.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78de600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d810b6c0 0x7f78d819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78de600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 0x7f78d81a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78de600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8115cd0 con 0x7f78d810b6c0 2024-09-21T01:12:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78de600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8115b50 con 0x7f78d8105d80 2024-09-21T01:12:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78de600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8115e50 con 0x7f78d810c010 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78cfe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d810b6c0 0x7f78d819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 0x7f78d81a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 0x7f78d81a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39862/0 (socket says 172.21.15.27:39862) 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78d7e00640 1 -- 172.21.15.27:0/979483737 learned_addr learned my addr 172.21.15.27:0/979483737 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78cfe00640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d810b6c0 0x7f78d819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46128/0 (socket says 172.21.15.27:46128) 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.360+0000 7f78d7400640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78d8105d80 0x7f78d819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d7e00640 1 -- 172.21.15.27:0/979483737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78d8105d80 msgr2=0x7f78d819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d7e00640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78d8105d80 0x7f78d819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d7e00640 1 -- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d810b6c0 msgr2=0x7f78d819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d7e00640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d810b6c0 0x7f78d819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d7e00640 1 -- 172.21.15.27:0/979483737 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81a2670 con 0x7f78d810c010 2024-09-21T01:12:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d7e00640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 0x7f78d81a1f40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78d000eb10 tx=0x7f78d000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0004b20 con 0x7f78d810c010 2024-09-21T01:12:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d0004cc0 con 0x7f78d810c010 2024-09-21T01:12:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81a2960 con 0x7f78d810c010 2024-09-21T01:12:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.361+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0010670 con 0x7f78d810c010 2024-09-21T01:12:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.362+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a90c0 con 0x7f78d810c010 2024-09-21T01:12:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.363+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d8106190 con 0x7f78d810c010 2024-09-21T01:12:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.364+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7f78d0005020 con 0x7f78d810c010 2024-09-21T01:12:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.365+0000 7f78d4c00640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f78bc07b020 0x7f78bc07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.365+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f78d009d410 con 0x7f78d810c010 2024-09-21T01:12:00.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.365+0000 7f78d7400640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f78bc07b020 0x7f78bc07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.366+0000 7f78d7400640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f78bc07b020 0x7f78bc07d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78c8031440 tx=0x7f78c8009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:00.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.370+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78d0069d80 con 0x7f78d810c010 2024-09-21T01:12:00.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.381+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f78d00615c0 con 0x7f78d810c010 2024-09-21T01:12:00.530 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:00.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:00.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: Standby manager daemon y started 2024-09-21T01:12:00.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:12:00.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:12:00.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:12:00.531 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:00 smithi027 ceph-mon[30375]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:12:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.588+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f78d810de10 con 0x7f78d810c010 2024-09-21T01:12:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.595+0000 7f78d4c00640 1 -- 172.21.15.27:0/979483737 <== mon.1 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v24)=0 v24) ==== 119+0+0 (secure 0 0 0) 0x7f78d00ad020 con 0x7f78d810c010 2024-09-21T01:12:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.597+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f78bc07b020 msgr2=0x7f78bc07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.597+0000 7f78de600640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f78bc07b020 0x7f78bc07d530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78c8031440 tx=0x7f78c8009560 comp rx=0 tx=0).stop 2024-09-21T01:12:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.597+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 msgr2=0x7f78d81a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.597+0000 7f78de600640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 0x7f78d81a1f40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f78d000eb10 tx=0x7f78d000c2d0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 shutdown_connections 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f78bc07b020 0x7f78bc07d530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d810c010 0x7f78d81a1f40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78d810b6c0 0x7f78d819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 --2- 172.21.15.27:0/979483737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78d8105d80 0x7f78d819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 >> 172.21.15.27:0/979483737 conn(0x7f78d81017f0 msgr2=0x7f78d8103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 shutdown_connections 2024-09-21T01:12:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.598+0000 7f78de600640 1 -- 172.21.15.27:0/979483737 wait complete. 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: Standby manager daemon y started 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:12:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:00 smithi098 ceph-mon[34372]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: Standby manager daemon y started 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T01:12:00.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:00 smithi089 ceph-mon[34486]: from='mgr.? 172.21.15.89:0/2653999694' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T01:12:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:01 smithi027 ceph-mon[30375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:01 smithi027 ceph-mon[30375]: mgrmap e20: x(active, since 20s), standbys: z, y 2024-09-21T01:12:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:12:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:01 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:01 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:01.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:01 smithi098 ceph-mon[34372]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:01.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:01 smithi098 ceph-mon[34372]: mgrmap e20: x(active, since 20s), standbys: z, y 2024-09-21T01:12:01.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:12:01.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:01 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:01.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:01 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:01.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:01 smithi089 ceph-mon[34486]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:01.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:01 smithi089 ceph-mon[34486]: mgrmap e20: x(active, since 20s), standbys: z, y 2024-09-21T01:12:01.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T01:12:01.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:01 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:01.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:01 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 -- 172.21.15.27:0/3153483332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570114570 msgr2=0x7fd570116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 --2- 172.21.15.27:0/3153483332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570114570 0x7fd570116990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd568009ca0 tx=0x7fd56802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 -- 172.21.15.27:0/3153483332 shutdown_connections 2024-09-21T01:12:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 --2- 172.21.15.27:0/3153483332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570114570 0x7fd570116990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 --2- 172.21.15.27:0/3153483332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd570078290 0x7fd570114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 --2- 172.21.15.27:0/3153483332 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd570077940 0x7fd570077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 -- 172.21.15.27:0/3153483332 >> 172.21.15.27:0/3153483332 conn(0x7fd570101680 msgr2=0x7fd570103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 -- 172.21.15.27:0/3153483332 shutdown_connections 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.439+0000 7fd575800640 1 -- 172.21.15.27:0/3153483332 wait complete. 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.440+0000 7fd575800640 1 Processor -- start 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.440+0000 7fd575800640 1 -- start start 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.440+0000 7fd575800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 0x7fd5701a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd575800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd570078290 0x7fd5701a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd575800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd570114570 0x7fd5701a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd575800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd570119f40 con 0x7fd570077940 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd575800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd570119dc0 con 0x7fd570114570 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd575800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd57011a0c0 con 0x7fd570078290 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 0x7fd5701a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:01.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 0x7fd5701a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46156/0 (socket says 172.21.15.27:46156) 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 -- 172.21.15.27:0/1146868853 learned_addr learned my addr 172.21.15.27:0/1146868853 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56f400640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd570078290 0x7fd5701a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd574e00640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd570114570 0x7fd5701a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 -- 172.21.15.27:0/1146868853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd570078290 msgr2=0x7fd5701a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd570078290 0x7fd5701a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 -- 172.21.15.27:0/1146868853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd570114570 msgr2=0x7fd5701a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd570114570 0x7fd5701a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.441+0000 7fd56fe00640 1 -- 172.21.15.27:0/1146868853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5701a68a0 con 0x7fd570077940 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.442+0000 7fd56fe00640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 0x7fd5701a0930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd56000e9f0 tx=0x7fd56000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.442+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56000cd80 con 0x7fd570077940 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.442+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5600108e0 con 0x7fd570077940 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.442+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560018980 con 0x7fd570077940 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.442+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5701a6b90 con 0x7fd570077940 2024-09-21T01:12:01.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.442+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5701ad2f0 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.443+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd570077d50 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.446+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fd560010430 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.447+0000 7fd56cc00640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5540b64d0 0x7fd5540b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.447+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd5600e5b00 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.447+0000 7fd56f400640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5540b64d0 0x7fd5540b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.448+0000 7fd56f400640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5540b64d0 0x7fd5540b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd57007a0c0 tx=0x7fd55c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.452+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5600b22d0 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.656+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd57002e640 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.662+0000 7fd56cc00640 1 -- 172.21.15.27:0/1146868853 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v26)=0 v26) ==== 133+0+0 (secure 0 0 0) 0x7fd5600b7210 con 0x7fd570077940 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5540b64d0 msgr2=0x7fd5540b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5540b64d0 0x7fd5540b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd57007a0c0 tx=0x7fd55c005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 msgr2=0x7fd5701a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 0x7fd5701a0930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd56000e9f0 tx=0x7fd56000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 shutdown_connections 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5540b64d0 0x7fd5540b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd570114570 0x7fd5701a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd570078290 0x7fd5701a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 --2- 172.21.15.27:0/1146868853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd570077940 0x7fd5701a0930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 >> 172.21.15.27:0/1146868853 conn(0x7fd570101680 msgr2=0x7fd570116560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 shutdown_connections 2024-09-21T01:12:01.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.665+0000 7fd575800640 1 -- 172.21.15.27:0/1146868853 wait complete. 2024-09-21T01:12:02.895 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:02 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1146868853' entity='client.admin' 2024-09-21T01:12:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:02 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1146868853' entity='client.admin' 2024-09-21T01:12:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:02 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1146868853' entity='client.admin' 2024-09-21T01:12:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 -- 172.21.15.27:0/1811542483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c1094c0 msgr2=0x7f5f6c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 --2- 172.21.15.27:0/1811542483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c1098d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009900 tx=0x7f5f5802a410 comp rx=0 tx=0).stop 2024-09-21T01:12:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 -- 172.21.15.27:0/1811542483 shutdown_connections 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 --2- 172.21.15.27:0/1811542483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c109e10 0x7f5f6c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 --2- 172.21.15.27:0/1811542483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c1098d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 --2- 172.21.15.27:0/1811542483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c103b80 0x7f5f6c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 -- 172.21.15.27:0/1811542483 >> 172.21.15.27:0/1811542483 conn(0x7f5f6c0ff590 msgr2=0x7f5f6c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 -- 172.21.15.27:0/1811542483 shutdown_connections 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.963+0000 7f5f73200640 1 -- 172.21.15.27:0/1811542483 wait complete. 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.964+0000 7f5f73200640 1 Processor -- start 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 -- start start 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c103b80 0x7f5f6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c109e10 0x7f5f6c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c113ad0 con 0x7f5f6c109e10 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c113950 con 0x7f5f6c1094c0 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f73200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c113c50 con 0x7f5f6c103b80 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f71200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c109e10 0x7f5f6c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.965+0000 7f5f6b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c103b80 0x7f5f6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54012/0 (socket says 172.21.15.27:54012) 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 -- 172.21.15.27:0/2203307182 learned_addr learned my addr 172.21.15.27:0/2203307182 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 -- 172.21.15.27:0/2203307182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c103b80 msgr2=0x7f5f6c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c103b80 0x7f5f6c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 -- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c109e10 msgr2=0x7f5f6c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c109e10 0x7f5f6c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 -- 172.21.15.27:0/2203307182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c1a8800 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.966+0000 7f5f6b400640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c075c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5f5802a930 tx=0x7f5f58031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.967+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5803f030 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.967+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f58004490 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.967+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58035070 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.967+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c1a8a90 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.967+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c1a8f20 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.969+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c103f90 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.970+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f5f58042030 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.972+0000 7f5f68c00640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f5f500b6550 0x7f5f500b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.972+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5f58105a70 con 0x7f5f6c1094c0 2024-09-21T01:12:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.972+0000 7f5f6be00640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f5f500b6550 0x7f5f500b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.973+0000 7f5f6be00640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f5f500b6550 0x7f5f500b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c077160 tx=0x7f5f5c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:00.976+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f580d2240 con 0x7f5f6c1094c0 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.181+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f5f6c1a9260 con 0x7f5f6c1094c0 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.187+0000 7f5f68c00640 1 -- 172.21.15.27:0/2203307182 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v25)=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f5f580d7180 con 0x7f5f6c1094c0 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f5f500b6550 msgr2=0x7f5f500b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f5f500b6550 0x7f5f500b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c077160 tx=0x7f5f5c005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 msgr2=0x7f5f6c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c075c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5f5802a930 tx=0x7f5f58031a20 comp rx=0 tx=0).stop 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 shutdown_connections 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f5f500b6550 0x7f5f500b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c109e10 0x7f5f6c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5f6c1094c0 0x7f5f6c075c50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 --2- 172.21.15.27:0/2203307182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f6c103b80 0x7f5f6c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:03.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 >> 172.21.15.27:0/2203307182 conn(0x7f5f6c0ff590 msgr2=0x7f5f6c107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:03.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 shutdown_connections 2024-09-21T01:12:03.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:01.190+0000 7f5f73200640 1 -- 172.21.15.27:0/2203307182 wait complete. 2024-09-21T01:12:03.917 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:03 smithi027 ceph-mon[30375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:03.917 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:03 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1515486009' entity='client.admin' 2024-09-21T01:12:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:03 smithi098 ceph-mon[34372]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:04.184 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:03 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1515486009' entity='client.admin' 2024-09-21T01:12:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.509+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3071229172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac110120 msgr2=0x7ff1ac112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.509+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3071229172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac110120 0x7ff1ac112540 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009ca0 tx=0x7ff1a402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3071229172 shutdown_connections 2024-09-21T01:12:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3071229172 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac110120 0x7ff1ac112540 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3071229172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3071229172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1ac075750 0x7ff1ac075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3071229172 >> 172.21.15.27:0/3071229172 conn(0x7ff1ac07b850 msgr2=0x7ff1ac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3071229172 shutdown_connections 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.510+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3071229172 wait complete. 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 Processor -- start 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 -- start start 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1ac075750 0x7ff1ac19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac110120 0x7ff1ac1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac115ce0 con 0x7ff1ac110120 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac115b60 con 0x7ff1ac0760a0 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.511+0000 7ff1b2c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac115e60 con 0x7ff1ac075750 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1abe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1ac075750 0x7ff1ac19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53986/0 (socket says 172.21.15.27:53986) 2024-09-21T01:12:04.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 -- 172.21.15.27:0/3243158130 learned_addr learned my addr 172.21.15.27:0/3243158130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1b0c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac110120 0x7ff1ac1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 -- 172.21.15.27:0/3243158130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1ac075750 msgr2=0x7ff1ac19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1ac075750 0x7ff1ac19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 -- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac110120 msgr2=0x7ff1ac1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac110120 0x7ff1ac1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 -- 172.21.15.27:0/3243158130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac1a2620 con 0x7ff1ac0760a0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1b0c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac110120 0x7ff1ac1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.512+0000 7ff1ab400640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac19cc50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e970 tx=0x7ff19800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.513+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19800cd80 con 0x7ff1ac0760a0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.513+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1980035f0 con 0x7ff1ac0760a0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.513+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198010690 con 0x7ff1ac0760a0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.513+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac1a2910 con 0x7ff1ac0760a0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.513+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac1a9070 con 0x7ff1ac0760a0 2024-09-21T01:12:04.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.514+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ac075b60 con 0x7ff1ac0760a0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.515+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 103194+0+0 (secure 0 0 0) 0x7ff198003aa0 con 0x7ff1ac0760a0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.517+0000 7ff1a8c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff19007b020 0x7ff19007d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.517+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff19809d0c0 con 0x7ff1ac0760a0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.517+0000 7ff1abe00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff19007b020 0x7ff19007d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.518+0000 7ff1abe00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff19007b020 0x7ff19007d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009bd0 tx=0x7ff19c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.521+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff198065650 con 0x7ff1ac0760a0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.722+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7ff1ac1a9370 con 0x7ff1ac0760a0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.723+0000 7ff1a8c00640 1 -- 172.21.15.27:0/3243158130 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v23)=0 v23) ==== 106+0+0 (secure 0 0 0) 0x7ff198069a30 con 0x7ff1ac0760a0 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff19007b020 msgr2=0x7ff19007d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff19007b020 0x7ff19007d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009bd0 tx=0x7ff19c005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 msgr2=0x7ff1ac19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac19cc50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e970 tx=0x7ff19800ee30 comp rx=0 tx=0).stop 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 shutdown_connections 2024-09-21T01:12:04.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff19007b020 0x7ff19007d530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1ac110120 0x7ff1ac1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1ac0760a0 0x7ff1ac19cc50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.725+0000 7ff1b2c00640 1 --2- 172.21.15.27:0/3243158130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1ac075750 0x7ff1ac19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.726+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 >> 172.21.15.27:0/3243158130 conn(0x7ff1ac07b850 msgr2=0x7ff1ac100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.726+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 shutdown_connections 2024-09-21T01:12:04.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:11:59.726+0000 7ff1b2c00640 1 -- 172.21.15.27:0/3243158130 wait complete. 2024-09-21T01:12:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:03 smithi089 ceph-mon[34486]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:03 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1515486009' entity='client.admin' 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 -- 172.21.15.27:0/1496039018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c103b70 msgr2=0x7fdf1c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 --2- 172.21.15.27:0/1496039018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c103b70 0x7fdf1c103f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 -- 172.21.15.27:0/1496039018 shutdown_connections 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 --2- 172.21.15.27:0/1496039018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c109e00 0x7fdf1c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 --2- 172.21.15.27:0/1496039018 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 --2- 172.21.15.27:0/1496039018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c103b70 0x7fdf1c103f80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 -- 172.21.15.27:0/1496039018 >> 172.21.15.27:0/1496039018 conn(0x7fdf1c0ff5e0 msgr2=0x7fdf1c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 -- 172.21.15.27:0/1496039018 shutdown_connections 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.172+0000 7fdf23800640 1 -- 172.21.15.27:0/1496039018 wait complete. 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.173+0000 7fdf23800640 1 Processor -- start 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.173+0000 7fdf23800640 1 -- start start 2024-09-21T01:12:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf23800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf1c103b70 0x7fdf1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf23800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf23800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c109e00 0x7fdf1c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf23800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113ac0 con 0x7fdf1c109e00 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf23800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113940 con 0x7fdf1c103b70 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf23800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c113c40 con 0x7fdf1c1094b0 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf21800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c109e00 0x7fdf1c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39944/0 (socket says 172.21.15.27:39944) 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 -- 172.21.15.27:0/3908924673 learned_addr learned my addr 172.21.15.27:0/3908924673 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf20e00640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf1c103b70 0x7fdf1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 -- 172.21.15.27:0/3908924673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf1c103b70 msgr2=0x7fdf1c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf1c103b70 0x7fdf1c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 -- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c109e00 msgr2=0x7fdf1c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c109e00 0x7fdf1c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.174+0000 7fdf1be00640 1 -- 172.21.15.27:0/3908924673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1a87b0 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.175+0000 7fdf1be00640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c075c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9f0 tx=0x7fdf0800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.175+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800ccc0 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.175+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf080108e0 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.175+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1a8aa0 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.175+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08018880 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.175+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a8fe0 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.177+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c103f80 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.182+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fdf08010430 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.184+0000 7fdf19600640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdf040b6550 0x7fdf040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.184+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdf080e5b30 con 0x7fdf1c1094b0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.184+0000 7fdf20e00640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdf040b6550 0x7fdf040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.185+0000 7fdf20e00640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdf040b6550 0x7fdf040b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf10000c00 tx=0x7fdf10031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.188+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf080adf20 con 0x7fdf1c1094b0 2024-09-21T01:12:05.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.395+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fdf1c02e640 con 0x7fdf1c1094b0 2024-09-21T01:12:05.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.404+0000 7fdf19600640 1 -- 172.21.15.27:0/3908924673 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v28)=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fdf080b2300 con 0x7fdf1c1094b0 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.406+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdf040b6550 msgr2=0x7fdf040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.406+0000 7fdf23800640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdf040b6550 0x7fdf040b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdf10000c00 tx=0x7fdf10031400 comp rx=0 tx=0).stop 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.406+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 msgr2=0x7fdf1c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c075c50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9f0 tx=0x7fdf0800edf0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 shutdown_connections 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdf040b6550 0x7fdf040b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf1c109e00 0x7fdf1c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf1c1094b0 0x7fdf1c075c50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 --2- 172.21.15.27:0/3908924673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdf1c103b70 0x7fdf1c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 >> 172.21.15.27:0/3908924673 conn(0x7fdf1c0ff5e0 msgr2=0x7fdf1c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:05.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 shutdown_connections 2024-09-21T01:12:05.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.407+0000 7fdf23800640 1 -- 172.21.15.27:0/3908924673 wait complete. 2024-09-21T01:12:06.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:05 smithi027 ceph-mon[30375]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:06.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:05 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:06.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:05 smithi098 ceph-mon[34372]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:05 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:06.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:05 smithi089 ceph-mon[34486]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:05 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:06.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 -- 172.21.15.27:0/720623870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 msgr2=0x7fb3bc103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:06.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/720623870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc103f90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009900 tx=0x7fb3b002a410 comp rx=0 tx=0).stop 2024-09-21T01:12:06.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 -- 172.21.15.27:0/720623870 shutdown_connections 2024-09-21T01:12:06.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/720623870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 0x7fb3bc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/720623870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/720623870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc103f90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 -- 172.21.15.27:0/720623870 >> 172.21.15.27:0/720623870 conn(0x7fb3bc0ff590 msgr2=0x7fb3bc1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.284+0000 7fb3c3e00640 1 -- 172.21.15.27:0/720623870 shutdown_connections 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.285+0000 7fb3c3e00640 1 -- 172.21.15.27:0/720623870 wait complete. 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.286+0000 7fb3c3e00640 1 Processor -- start 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.286+0000 7fb3c3e00640 1 -- start start 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.286+0000 7fb3c3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 0x7fb3bc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c3e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113b20 con 0x7fb3bc1094c0 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c3e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1139a0 con 0x7fb3bc109e10 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c3e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113ca0 con 0x7fb3bc103b80 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:06.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46202/0 (socket says 172.21.15.27:46202) 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c0a00640 1 -- 172.21.15.27:0/4164292500 learned_addr learned my addr 172.21.15.27:0/4164292500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c1400640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c1e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 0x7fb3bc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c1e00640 1 -- 172.21.15.27:0/4164292500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 msgr2=0x7fb3bc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.287+0000 7fb3c1e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c1e00640 1 -- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 msgr2=0x7fb3bc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c1e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c1e00640 1 -- 172.21.15.27:0/4164292500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc1a04a0 con 0x7fb3bc109e10 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c0a00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c1400640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c1e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 0x7fb3bc19fd70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009fd0 tx=0x7fb3b400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400ce20 con 0x7fb3bc109e10 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b4005020 con 0x7fb3bc109e10 2024-09-21T01:12:06.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4010690 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1a0790 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.288+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc0755f0 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.289+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc103f90 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.294+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fb3b40054d0 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.295+0000 7fb3ba000640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb3a40b6550 0x7fb3a40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.296+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb3b40e5d00 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.296+0000 7fb3c1400640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb3a40b6550 0x7fb3a40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.296+0000 7fb3c1400640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb3a40b6550 0x7fb3a40b8a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005ab0 tx=0x7fb3b003c040 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.300+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb3b40ae190 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.516+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fb3bc02e640 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.517+0000 7fb3ba000640 1 -- 172.21.15.27:0/4164292500 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v28)=0 v28) ==== 108+0+0 (secure 0 0 0) 0x7fb3b40b2570 con 0x7fb3bc109e10 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb3a40b6550 msgr2=0x7fb3a40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:06.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb3a40b6550 0x7fb3a40b8a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005ab0 tx=0x7fb3b003c040 comp rx=0 tx=0).stop 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 msgr2=0x7fb3bc19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 0x7fb3bc19fd70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009fd0 tx=0x7fb3b400ef10 comp rx=0 tx=0).stop 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 shutdown_connections 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb3a40b6550 0x7fb3a40b8a60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3bc109e10 0x7fb3bc19fd70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3bc1094c0 0x7fb3bc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 --2- 172.21.15.27:0/4164292500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3bc103b80 0x7fb3bc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 >> 172.21.15.27:0/4164292500 conn(0x7fb3bc0ff590 msgr2=0x7fb3bc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 shutdown_connections 2024-09-21T01:12:06.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:05.520+0000 7fb3c3e00640 1 -- 172.21.15.27:0/4164292500 wait complete. 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 -- 172.21.15.27:0/3706796957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 msgr2=0x7fc500112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 --2- 172.21.15.27:0/3706796957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc500112520 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009ca0 tx=0x7fc4f802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 -- 172.21.15.27:0/3706796957 shutdown_connections 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 --2- 172.21.15.27:0/3706796957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc500112520 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 --2- 172.21.15.27:0/3706796957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 0x7fc50010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 --2- 172.21.15.27:0/3706796957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5000757b0 0x7fc500075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 -- 172.21.15.27:0/3706796957 >> 172.21.15.27:0/3706796957 conn(0x7fc5000ff590 msgr2=0x7fc5001019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 -- 172.21.15.27:0/3706796957 shutdown_connections 2024-09-21T01:12:07.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.382+0000 7fc506000640 1 -- 172.21.15.27:0/3706796957 wait complete. 2024-09-21T01:12:07.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.383+0000 7fc506000640 1 Processor -- start 2024-09-21T01:12:07.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.383+0000 7fc506000640 1 -- start start 2024-09-21T01:12:07.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc506000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5000757b0 0x7fc50019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:07.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc506000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 0x7fc50019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc506000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc5001a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc506000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc500115ce0 con 0x7fc5001100e0 2024-09-21T01:12:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc506000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc500115b60 con 0x7fc5000757b0 2024-09-21T01:12:07.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc506000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc500115e60 con 0x7fc500076100 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc5001a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc5001a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36538/0 (socket says 172.21.15.27:36538) 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc4ffe00640 1 -- 172.21.15.27:0/3589462868 learned_addr learned my addr 172.21.15.27:0/3589462868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc4ff400640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5000757b0 0x7fc50019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.384+0000 7fc4f7e00640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 0x7fc50019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4f7e00640 1 -- 172.21.15.27:0/3589462868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5000757b0 msgr2=0x7fc50019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4f7e00640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5000757b0 0x7fc50019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4f7e00640 1 -- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 msgr2=0x7fc5001a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:07.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4f7e00640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc5001a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4f7e00640 1 -- 172.21.15.27:0/3589462868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5001a25c0 con 0x7fc500076100 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4f7e00640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 0x7fc50019cbf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800e9c0 tx=0x7fc4e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800ccc0 con 0x7fc500076100 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e80035f0 con 0x7fc500076100 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800ce60 con 0x7fc500076100 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.385+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5001a28b0 con 0x7fc500076100 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.386+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5001a9010 con 0x7fc500076100 2024-09-21T01:12:07.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.387+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc500075bc0 con 0x7fc500076100 2024-09-21T01:12:07.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.393+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fc4e8004000 con 0x7fc500076100 2024-09-21T01:12:07.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.394+0000 7fc4fcc00640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4e40b65d0 0x7fc4e40b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:07.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.395+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc4e80e5b90 con 0x7fc500076100 2024-09-21T01:12:07.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.395+0000 7fc4ff400640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4e40b65d0 0x7fc4e40b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:07.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.395+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc4e8116710 con 0x7fc500076100 2024-09-21T01:12:07.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.396+0000 7fc4ff400640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4e40b65d0 0x7fc4e40b8ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009bd0 tx=0x7fc4f0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.598+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fc5001a9370 con 0x7fc500076100 2024-09-21T01:12:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.605+0000 7fc4fcc00640 1 -- 172.21.15.27:0/3589462868 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7fc4e80adf80 con 0x7fc500076100 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4e40b65d0 msgr2=0x7fc4e40b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4e40b65d0 0x7fc4e40b8ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009bd0 tx=0x7fc4f0005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 msgr2=0x7fc50019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 0x7fc50019cbf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800e9c0 tx=0x7fc4e800ee80 comp rx=0 tx=0).stop 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 shutdown_connections 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4e40b65d0 0x7fc4e40b8ae0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5001100e0 0x7fc5001a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc500076100 0x7fc50019cbf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 --2- 172.21.15.27:0/3589462868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc5000757b0 0x7fc50019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 >> 172.21.15.27:0/3589462868 conn(0x7fc5000ff590 msgr2=0x7fc500101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 shutdown_connections 2024-09-21T01:12:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.608+0000 7fc506000640 1 -- 172.21.15.27:0/3589462868 wait complete. 2024-09-21T01:12:07.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:07 smithi027 ceph-mon[30375]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:07.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:07 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:07.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:07 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:07 smithi098 ceph-mon[34372]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:07 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:07 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:08.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:07 smithi089 ceph-mon[34486]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:07 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:07 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:08.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.489+0000 7f7ec6800640 1 -- 172.21.15.27:0/2001282878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01092f0 msgr2=0x7f7ec0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.489+0000 7f7ec6800640 1 --2- 172.21.15.27:0/2001282878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01092f0 0x7f7ec0109700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7eac009900 tx=0x7f7eac02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 -- 172.21.15.27:0/2001282878 shutdown_connections 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 --2- 172.21.15.27:0/2001282878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec0109c40 0x7f7ec0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 --2- 172.21.15.27:0/2001282878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01092f0 0x7f7ec0109700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 --2- 172.21.15.27:0/2001282878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01039b0 0x7f7ec0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 -- 172.21.15.27:0/2001282878 >> 172.21.15.27:0/2001282878 conn(0x7f7ec00ff440 msgr2=0x7f7ec0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 -- 172.21.15.27:0/2001282878 shutdown_connections 2024-09-21T01:12:08.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.490+0000 7f7ec6800640 1 -- 172.21.15.27:0/2001282878 wait complete. 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 Processor -- start 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 -- start start 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 0x7f7ec019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 0x7f7ec019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec0109c40 0x7f7ec019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0113950 con 0x7f7ec01039b0 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01137d0 con 0x7f7ec01092f0 2024-09-21T01:12:08.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.491+0000 7f7ec6800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0113ad0 con 0x7f7ec0109c40 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 0x7f7ec019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 0x7f7ec019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 0x7f7ec019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43664/0 (socket says 172.21.15.27:43664) 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 0x7f7ec019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36558/0 (socket says 172.21.15.27:36558) 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 -- 172.21.15.27:0/486325638 learned_addr learned my addr 172.21.15.27:0/486325638 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec5e00640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec0109c40 0x7f7ec019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 -- 172.21.15.27:0/486325638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec0109c40 msgr2=0x7f7ec019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:08.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec0109c40 0x7f7ec019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 -- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 msgr2=0x7f7ec019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 0x7f7ec019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec4a00640 1 -- 172.21.15.27:0/486325638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec01a0270 con 0x7f7ec01092f0 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.492+0000 7f7ec5400640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 0x7f7ec019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.493+0000 7f7ec4a00640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 0x7f7ec019a8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005ab0 tx=0x7f7eac003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.493+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac03f070 con 0x7f7ec01092f0 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.493+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eac031b50 con 0x7f7ec01092f0 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.493+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec01a0500 con 0x7f7ec01092f0 2024-09-21T01:12:08.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.493+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec01a6cc0 con 0x7f7ec01092f0 2024-09-21T01:12:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.493+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac035030 con 0x7f7ec01092f0 2024-09-21T01:12:08.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.494+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec0103dc0 con 0x7f7ec01092f0 2024-09-21T01:12:08.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.496+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f7eac042070 con 0x7f7ec01092f0 2024-09-21T01:12:08.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.498+0000 7f7ebe000640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ea40b6550 0x7f7ea40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:08.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.498+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7eac106140 con 0x7f7ec01092f0 2024-09-21T01:12:08.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.498+0000 7f7ec5400640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ea40b6550 0x7f7ea40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:08.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.499+0000 7f7ec5400640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ea40b6550 0x7f7ea40b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00095f0 tx=0x7f7eb0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:08.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.500+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7eac0d2910 con 0x7f7ec01092f0 2024-09-21T01:12:08.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.717+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f7ec010ba60 con 0x7f7ec01092f0 2024-09-21T01:12:08.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.723+0000 7f7ebe000640 1 -- 172.21.15.27:0/486325638 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f7eac0d7850 con 0x7f7ec01092f0 2024-09-21T01:12:08.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.725+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ea40b6550 msgr2=0x7f7ea40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:08.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.725+0000 7f7ec6800640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ea40b6550 0x7f7ea40b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00095f0 tx=0x7f7eb0005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 msgr2=0x7f7ec019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 0x7f7ec019a8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7eac005ab0 tx=0x7f7eac003b90 comp rx=0 tx=0).stop 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 shutdown_connections 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7ea40b6550 0x7f7ea40b8a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ec0109c40 0x7f7ec019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ec01092f0 0x7f7ec019a8a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 --2- 172.21.15.27:0/486325638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ec01039b0 0x7f7ec019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 >> 172.21.15.27:0/486325638 conn(0x7f7ec00ff440 msgr2=0x7f7ec0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 shutdown_connections 2024-09-21T01:12:08.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.726+0000 7f7ec6800640 1 -- 172.21.15.27:0/486325638 wait complete. 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 -- 172.21.15.27:0/212885166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 msgr2=0x7f1904112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 --2- 172.21.15.27:0/212885166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f1904112840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009ca0 tx=0x7f18fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 -- 172.21.15.27:0/212885166 shutdown_connections 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 --2- 172.21.15.27:0/212885166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f1904112840 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 --2- 172.21.15.27:0/212885166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f190410b6c0 0x7f190410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 --2- 172.21.15.27:0/212885166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1904105d80 0x7f1904106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 -- 172.21.15.27:0/212885166 >> 172.21.15.27:0/212885166 conn(0x7f1904101810 msgr2=0x7f1904103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 -- 172.21.15.27:0/212885166 shutdown_connections 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.987+0000 7f190ae00640 1 -- 172.21.15.27:0/212885166 wait complete. 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.988+0000 7f190ae00640 1 Processor -- start 2024-09-21T01:12:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.988+0000 7f190ae00640 1 -- start start 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f190ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 0x7f190419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f190ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190410b6c0 0x7f190419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f190ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f19041a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f190ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1904115d20 con 0x7f1904105d80 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f190ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1904115ba0 con 0x7f190410c010 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f190ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1904115ea0 con 0x7f190410b6c0 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f1903e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 0x7f190419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f1903e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 0x7f190419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36588/0 (socket says 172.21.15.27:36588) 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f1903e00640 1 -- 172.21.15.27:0/3630037235 learned_addr learned my addr 172.21.15.27:0/3630037235 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f1908e00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f19041a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.989+0000 7f18fbe00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190410b6c0 0x7f190419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1903e00640 1 -- 172.21.15.27:0/3630037235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190410b6c0 msgr2=0x7f190419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1903e00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190410b6c0 0x7f190419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1903e00640 1 -- 172.21.15.27:0/3630037235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 msgr2=0x7f19041a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1903e00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f19041a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1903e00640 1 -- 172.21.15.27:0/3630037235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041a25e0 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1908e00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f19041a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1903e00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 0x7f190419c670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400cd80 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f40054d0 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19041a28d0 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.990+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4010690 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.991+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19041a9030 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.992+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1904106190 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.994+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f18f4004e30 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.996+0000 7f1901600640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f18e80b64d0 0x7f18e80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.996+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f18f40e5d80 con 0x7f1904105d80 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.996+0000 7f18fbe00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f18e80b64d0 0x7f18e80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.997+0000 7f18fbe00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f18e80b64d0 0x7f18e80b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1904105060 tx=0x7f18ec005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.998+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18f40ae170 con 0x7f1904105d80 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 -- 172.21.15.27:0/2313691611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24181047c0 msgr2=0x7f2418104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 --2- 172.21.15.27:0/2313691611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24181047c0 0x7f2418104bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f240c009900 tx=0x7f240c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 -- 172.21.15.27:0/2313691611 shutdown_connections 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 --2- 172.21.15.27:0/2313691611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 0x7f2418110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 --2- 172.21.15.27:0/2313691611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2418105110 0x7f241810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 --2- 172.21.15.27:0/2313691611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24181047c0 0x7f2418104bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 -- 172.21.15.27:0/2313691611 >> 172.21.15.27:0/2313691611 conn(0x7f24180fe2c0 msgr2=0x7f2418100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 -- 172.21.15.27:0/2313691611 shutdown_connections 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.895+0000 7f241e200640 1 -- 172.21.15.27:0/2313691611 wait complete. 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.897+0000 7f241e200640 1 Processor -- start 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.897+0000 7f241e200640 1 -- start start 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 0x7f241819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418105110 0x7f241819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241e200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 0x7f24181a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241e200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2418113880 con 0x7f2418105110 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2418113700 con 0x7f24181047c0 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241e200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2418113a00 con 0x7f241810dc20 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 0x7f241819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241ce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 0x7f241819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43676/0 (socket says 172.21.15.27:43676) 2024-09-21T01:12:09.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241ce00640 1 -- 172.21.15.27:0/1837216621 learned_addr learned my addr 172.21.15.27:0/1837216621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f2417e00640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418105110 0x7f241819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.898+0000 7f241d800640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 0x7f24181a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241ce00640 1 -- 172.21.15.27:0/1837216621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 msgr2=0x7f24181a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241ce00640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 0x7f24181a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241ce00640 1 -- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418105110 msgr2=0x7f241819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241ce00640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418105110 0x7f241819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241ce00640 1 -- 172.21.15.27:0/1837216621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24181a24e0 con 0x7f24181047c0 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241d800640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 0x7f24181a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241ce00640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 0x7f241819c570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f240c005940 tx=0x7f240c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c03f030 con 0x7f24181047c0 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c003aa0 con 0x7f24181047c0 2024-09-21T01:12:09.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c035070 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.899+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24181a2770 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.900+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24181a8f30 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.901+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241806b560 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.904+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f240c042030 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.907+0000 7f2415600640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24000b6550 0x7f24000b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.907+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f240c106040 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.907+0000 7f2417e00640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24000b6550 0x7f24000b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.907+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f240c106450 con 0x7f24181047c0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:08.908+0000 7f2417e00640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24000b6550 0x7f24000b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2418101b70 tx=0x7f2404005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.115+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f241819d750 con 0x7f24181047c0 2024-09-21T01:12:09.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.124+0000 7f2415600640 1 -- 172.21.15.27:0/1837216621 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v32)=0 v32) ==== 120+0+0 (secure 0 0 0) 0x7f240c0ce430 con 0x7f24181047c0 2024-09-21T01:12:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24000b6550 msgr2=0x7f24000b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24000b6550 0x7f24000b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2418101b70 tx=0x7f2404005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 msgr2=0x7f241819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 0x7f241819c570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f240c005940 tx=0x7f240c004270 comp rx=0 tx=0).stop 2024-09-21T01:12:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 shutdown_connections 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24000b6550 0x7f24000b8a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241810dc20 0x7f24181a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2418105110 0x7f241819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 --2- 172.21.15.27:0/1837216621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24181047c0 0x7f241819c570 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 >> 172.21.15.27:0/1837216621 conn(0x7f24180fe2c0 msgr2=0x7f2418102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.126+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 shutdown_connections 2024-09-21T01:12:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.127+0000 7f241e200640 1 -- 172.21.15.27:0/1837216621 wait complete. 2024-09-21T01:12:09.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.203+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f190402e640 con 0x7f1904105d80 2024-09-21T01:12:09.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.209+0000 7f1901600640 1 -- 172.21.15.27:0/3630037235 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v33)=0 v33) ==== 119+0+0 (secure 0 0 0) 0x7f18f40b2550 con 0x7f1904105d80 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f18e80b64d0 msgr2=0x7f18e80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f18e80b64d0 0x7f18e80b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1904105060 tx=0x7f18ec005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 msgr2=0x7f190419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 0x7f190419c670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400edf0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 shutdown_connections 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f18e80b64d0 0x7f18e80b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f190410c010 0x7f19041a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190410b6c0 0x7f190419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 --2- 172.21.15.27:0/3630037235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1904105d80 0x7f190419c670 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 >> 172.21.15.27:0/3630037235 conn(0x7f1904101810 msgr2=0x7f1904108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 shutdown_connections 2024-09-21T01:12:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:09.212+0000 7f190ae00640 1 -- 172.21.15.27:0/3630037235 wait complete. 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 -- 172.21.15.27:0/3762107465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c410c010 msgr2=0x7f17c4112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 --2- 172.21.15.27:0/3762107465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c410c010 0x7f17c4112840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009ca0 tx=0x7f17bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 -- 172.21.15.27:0/3762107465 shutdown_connections 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 --2- 172.21.15.27:0/3762107465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c410c010 0x7f17c4112840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 --2- 172.21.15.27:0/3762107465 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410b6c0 0x7f17c410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 --2- 172.21.15.27:0/3762107465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c4105d80 0x7f17c4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.551+0000 7f17c9000640 1 -- 172.21.15.27:0/3762107465 >> 172.21.15.27:0/3762107465 conn(0x7f17c4101810 msgr2=0x7f17c4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.552+0000 7f17c9000640 1 -- 172.21.15.27:0/3762107465 shutdown_connections 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.552+0000 7f17c9000640 1 -- 172.21.15.27:0/3762107465 wait complete. 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 Processor -- start 2024-09-21T01:12:09.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 -- start start 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c4105d80 0x7f17c419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 0x7f17c419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410c010 0x7f17c41a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4115d20 con 0x7f17c4105d80 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4115ba0 con 0x7f17c410c010 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c9000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4115ea0 con 0x7f17c410b6c0 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 0x7f17c419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.553+0000 7f17c1e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c4105d80 0x7f17c419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c1400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 0x7f17c419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54140/0 (socket says 172.21.15.27:54140) 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c1400640 1 -- 172.21.15.27:0/335868217 learned_addr learned my addr 172.21.15.27:0/335868217 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410c010 0x7f17c41a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 -- 172.21.15.27:0/335868217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 msgr2=0x7f17c419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 0x7f17c419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 -- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c4105d80 msgr2=0x7f17c419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c4105d80 0x7f17c419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 -- 172.21.15.27:0/335868217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c41a25e0 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c1e00640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c4105d80 0x7f17c419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c1400640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 0x7f17c419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.554+0000 7f17c2800640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410c010 0x7f17c41a1eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005e30 tx=0x7f17bc0317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.555+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc003b70 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.555+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41a2870 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.555+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17bc003d10 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.555+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc02f970 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.555+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a9030 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.556+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c4106190 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.558+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f17bc02fb10 con 0x7f17c410c010 2024-09-21T01:12:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.560+0000 7f17baa00640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f17a80b64d0 0x7f17a80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.560+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f17bc106760 con 0x7f17c410c010 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.560+0000 7f17c1e00640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f17a80b64d0 0x7f17a80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.561+0000 7f17c1e00640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f17a80b64d0 0x7f17a80b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009fc0 tx=0x7f17b4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.564+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17bc0ceb50 con 0x7f17c410c010 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.772+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f17c410de60 con 0x7f17c410c010 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.780+0000 7f17baa00640 1 -- 172.21.15.27:0/335868217 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v30)=0 v30) ==== 125+0+0 (secure 0 0 0) 0x7f17bc0d2f30 con 0x7f17c410c010 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f17a80b64d0 msgr2=0x7f17a80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f17a80b64d0 0x7f17a80b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009fc0 tx=0x7f17b4005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410c010 msgr2=0x7f17c41a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410c010 0x7f17c41a1eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005e30 tx=0x7f17bc0317c0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 shutdown_connections 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f17a80b64d0 0x7f17a80b89e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f17c410c010 0x7f17c41a1eb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17c410b6c0 0x7f17c419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 --2- 172.21.15.27:0/335868217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c4105d80 0x7f17c419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.783+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 >> 172.21.15.27:0/335868217 conn(0x7f17c4101810 msgr2=0x7f17c4108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.784+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 shutdown_connections 2024-09-21T01:12:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:07.784+0000 7f17c9000640 1 -- 172.21.15.27:0/335868217 wait complete. 2024-09-21T01:12:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 -- 172.21.15.27:0/3357446694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9074109df0 msgr2=0x7f9074110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 --2- 172.21.15.27:0/3357446694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9074109df0 0x7f9074110620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f906c009900 tx=0x7f906c02a410 comp rx=0 tx=0).stop 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 -- 172.21.15.27:0/3357446694 shutdown_connections 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 --2- 172.21.15.27:0/3357446694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9074109df0 0x7f9074110620 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 --2- 172.21.15.27:0/3357446694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90741094a0 0x7f90741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 --2- 172.21.15.27:0/3357446694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074103b60 0x7f9074103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 -- 172.21.15.27:0/3357446694 >> 172.21.15.27:0/3357446694 conn(0x7f907407b850 msgr2=0x7f907407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 -- 172.21.15.27:0/3357446694 shutdown_connections 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.611+0000 7f907ae00640 1 -- 172.21.15.27:0/3357446694 wait complete. 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.612+0000 7f907ae00640 1 Processor -- start 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 -- start start 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9074103b60 0x7f907419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 0x7f907419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074109df0 0x7f90741a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9074113b00 con 0x7f90741094a0 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9074113980 con 0x7f9074109df0 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f907ae00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9074113c80 con 0x7f9074103b60 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f906b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 0x7f907419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f906b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 0x7f907419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46166/0 (socket says 172.21.15.27:46166) 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f906b400640 1 -- 172.21.15.27:0/1515486009 learned_addr learned my addr 172.21.15.27:0/1515486009 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f9078e00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074109df0 0x7f90741a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.613+0000 7f9073e00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9074103b60 0x7f907419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f906b400640 1 -- 172.21.15.27:0/1515486009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9074103b60 msgr2=0x7f907419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f906b400640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9074103b60 0x7f907419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f906b400640 1 -- 172.21.15.27:0/1515486009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074109df0 msgr2=0x7f90741a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f906b400640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074109df0 0x7f90741a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f906b400640 1 -- 172.21.15.27:0/1515486009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90741a2580 con 0x7f90741094a0 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f9078e00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074109df0 0x7f90741a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f906b400640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 0x7f907419cbb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f905c00e9f0 tx=0x7f905c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c00ccc0 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c0108e0 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741a2870 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.614+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c018860 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.615+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a8fd0 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.616+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9074103f70 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.618+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f905c010430 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.619+0000 7f9071600640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90580b6550 0x7f90580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.620+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f905c0e59d0 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.620+0000 7f9073e00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90580b6550 0x7f90580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.620+0000 7f9073e00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90580b6550 0x7f90580b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9064005fd0 tx=0x7f9064009500 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.623+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f905c0b21a0 con 0x7f90741094a0 2024-09-21T01:12:09.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.830+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f90741a92b0 con 0x7f90741094a0 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.835+0000 7f9071600640 1 -- 172.21.15.27:0/1515486009 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7f905c0b70e0 con 0x7f90741094a0 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90580b6550 msgr2=0x7f90580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90580b6550 0x7f90580b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9064005fd0 tx=0x7f9064009500 comp rx=0 tx=0).stop 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 msgr2=0x7f907419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 0x7f907419cbb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f905c00e9f0 tx=0x7f905c00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 shutdown_connections 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90580b6550 0x7f90580b8a60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9074109df0 0x7f90741a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90741094a0 0x7f907419cbb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 --2- 172.21.15.27:0/1515486009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9074103b60 0x7f907419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 >> 172.21.15.27:0/1515486009 conn(0x7f907407b850 msgr2=0x7f9074105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 shutdown_connections 2024-09-21T01:12:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:02.838+0000 7f907ae00640 1 -- 172.21.15.27:0/1515486009 wait complete. 2024-09-21T01:12:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:09 smithi027 ceph-mon[30375]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:09 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:09 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:09 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3630037235' entity='client.admin' 2024-09-21T01:12:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:09 smithi089 ceph-mon[34486]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:09 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:09 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:09 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3630037235' entity='client.admin' 2024-09-21T01:12:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:09 smithi098 ceph-mon[34372]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:09 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:09 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:09 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3630037235' entity='client.admin' 2024-09-21T01:12:10.418 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:12:11.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:11 smithi027 ceph-mon[30375]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:11.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:11.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:11.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:12:11.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:12:11.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:11 smithi089 ceph-mon[34486]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:12:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:12:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 -- 172.21.15.27:0/2982857361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 msgr2=0x7fbaac109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 --2- 172.21.15.27:0/2982857361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 0x7fbaac109700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fba98009860 tx=0x7fba9802a340 comp rx=0 tx=0).stop 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 -- 172.21.15.27:0/2982857361 shutdown_connections 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 --2- 172.21.15.27:0/2982857361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaac109c40 0x7fbaac110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 --2- 172.21.15.27:0/2982857361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 0x7fbaac109700 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 --2- 172.21.15.27:0/2982857361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 -- 172.21.15.27:0/2982857361 >> 172.21.15.27:0/2982857361 conn(0x7fbaac0ff440 msgr2=0x7fbaac101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 -- 172.21.15.27:0/2982857361 shutdown_connections 2024-09-21T01:12:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.126+0000 7fbab2400640 1 -- 172.21.15.27:0/2982857361 wait complete. 2024-09-21T01:12:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.127+0000 7fbab2400640 1 Processor -- start 2024-09-21T01:12:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 -- start start 2024-09-21T01:12:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 0x7fbaac19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaac109c40 0x7fbaac19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac113900 con 0x7fbaac1039b0 2024-09-21T01:12:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac113780 con 0x7fbaac109c40 2024-09-21T01:12:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab2400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac113a80 con 0x7fbaac1092f0 2024-09-21T01:12:12.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:12.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab1000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36612/0 (socket says 172.21.15.27:36612) 2024-09-21T01:12:12.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab1000640 1 -- 172.21.15.27:0/639542645 learned_addr learned my addr 172.21.15.27:0/639542645 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:12.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbaabe00640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 0x7fbaac19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:12.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.128+0000 7fbab1a00640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaac109c40 0x7fbaac19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbab1000640 1 -- 172.21.15.27:0/639542645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 msgr2=0x7fbaac19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbab1000640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 0x7fbaac19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbab1000640 1 -- 172.21.15.27:0/639542645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaac109c40 msgr2=0x7fbaac19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbab1000640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaac109c40 0x7fbaac19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbab1000640 1 -- 172.21.15.27:0/639542645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac1a0300 con 0x7fbaac1039b0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbab1000640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac19a390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000e9c0 tx=0x7fbaa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.129+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa000cde0 con 0x7fbaac1039b0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.130+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac1a05f0 con 0x7fbaac1039b0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.130+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa00035f0 con 0x7fbaac1039b0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.130+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0010690 con 0x7fbaac1039b0 2024-09-21T01:12:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.130+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac1a6d50 con 0x7fbaac1039b0 2024-09-21T01:12:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.131+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac103dc0 con 0x7fbaac1039b0 2024-09-21T01:12:12.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.133+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fbaa0004000 con 0x7fbaac1039b0 2024-09-21T01:12:12.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.135+0000 7fbaa9600640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba940b65d0 0x7fba940b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.135+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbaa00e5a70 con 0x7fbaac1039b0 2024-09-21T01:12:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.135+0000 7fbaabe00640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba940b65d0 0x7fba940b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:12.138 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.136+0000 7fbaabe00640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba940b65d0 0x7fba940b8ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba98005940 tx=0x7fba980058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:12.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.139+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbaa0009510 con 0x7fbaac1039b0 2024-09-21T01:12:12.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:11 smithi098 ceph-mon[34372]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:12.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T01:12:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T01:12:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.340+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fbaac19b4c0 con 0x7fbaac1039b0 2024-09-21T01:12:12.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.345+0000 7fbaa9600640 1 -- 172.21.15.27:0/639542645 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v34)=0 v34) ==== 127+0+0 (secure 0 0 0) 0x7fbaa00ade60 con 0x7fbaac1039b0 2024-09-21T01:12:12.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba940b65d0 msgr2=0x7fba940b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:12.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba940b65d0 0x7fba940b8ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba98005940 tx=0x7fba980058d0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 msgr2=0x7fbaac19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:12.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac19a390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000e9c0 tx=0x7fbaa000ee80 comp rx=0 tx=0).stop 2024-09-21T01:12:12.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 shutdown_connections 2024-09-21T01:12:12.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba940b65d0 0x7fba940b8ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbaac109c40 0x7fbaac19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbaac1092f0 0x7fbaac19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 --2- 172.21.15.27:0/639542645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaac1039b0 0x7fbaac19a390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:12.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 >> 172.21.15.27:0/639542645 conn(0x7fbaac0ff440 msgr2=0x7fbaac101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:12.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 shutdown_connections 2024-09-21T01:12:12.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:12.347+0000 7fbab2400640 1 -- 172.21.15.27:0/639542645 wait complete. 2024-09-21T01:12:12.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:12:12.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: Reconfiguring daemon mon.c on smithi098 2024-09-21T01:12:12.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/639542645' entity='client.admin' 2024-09-21T01:12:12.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:12.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:12.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:12.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:12.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:12.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: Reconfiguring daemon mon.c on smithi098 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/639542645' entity='client.admin' 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:13.121 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: Reconfiguring mon.c (monmap changed)... 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: Reconfiguring daemon mon.c on smithi098 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/639542645' entity='client.admin' 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:13.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:14.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:13 smithi089 ceph-mon[34486]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:14.020 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:13 smithi027 ceph-mon[30375]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:14.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:13 smithi098 ceph-mon[34372]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:15.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:15 smithi089 ceph-mon[34486]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:16.016 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:15 smithi027 ceph-mon[30375]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:15 smithi098 ceph-mon[34372]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.385+0000 7f29c0e00640 1 -- 172.21.15.27:0/228749485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc109df0 msgr2=0x7f29bc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.385+0000 7f29c0e00640 1 --2- 172.21.15.27:0/228749485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc109df0 0x7f29bc110620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009be0 tx=0x7f29b402a6c0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 -- 172.21.15.27:0/228749485 shutdown_connections 2024-09-21T01:12:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 --2- 172.21.15.27:0/228749485 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc109df0 0x7f29bc110620 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 --2- 172.21.15.27:0/228749485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc1094a0 0x7f29bc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.620 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 --2- 172.21.15.27:0/228749485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc103b60 0x7f29bc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 -- 172.21.15.27:0/228749485 >> 172.21.15.27:0/228749485 conn(0x7f29bc07b850 msgr2=0x7f29bc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 -- 172.21.15.27:0/228749485 shutdown_connections 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.386+0000 7f29c0e00640 1 -- 172.21.15.27:0/228749485 wait complete. 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 Processor -- start 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 -- start start 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 0x7f29bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc1094a0 0x7f29bc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc109df0 0x7f29bc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113ab0 con 0x7f29bc109df0 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113930 con 0x7f29bc1094a0 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.387+0000 7f29c0e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc113c30 con 0x7f29bc103b60 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 0x7f29bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29ba800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc109df0 0x7f29bc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 0x7f29bc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47646/0 (socket says 172.21.15.27:47646) 2024-09-21T01:12:16.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 -- 172.21.15.27:0/1678775992 learned_addr learned my addr 172.21.15.27:0/1678775992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b3e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc1094a0 0x7f29bc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 -- 172.21.15.27:0/1678775992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc1094a0 msgr2=0x7f29bc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc1094a0 0x7f29bc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 -- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc109df0 msgr2=0x7f29bc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc109df0 0x7f29bc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 -- 172.21.15.27:0/1678775992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc1a86e0 con 0x7f29bc103b60 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29ba800640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc109df0 0x7f29bc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.388+0000 7f29b9e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 0x7f29bc075710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e9c0 tx=0x7f29ac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.389+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00cde0 con 0x7f29bc103b60 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.389+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac0035f0 con 0x7f29bc103b60 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.389+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc1a89d0 con 0x7f29bc103b60 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.389+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac010690 con 0x7f29bc103b60 2024-09-21T01:12:16.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.389+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a8f60 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.390+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc103f70 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.392+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f29ac004000 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.394+0000 7f29b2a00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f29a00b6550 0x7f29a00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.394+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f29ac0e5a70 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.394+0000 7f29b3e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f29a00b6550 0x7f29a00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.395+0000 7f29b3e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f29a00b6550 0x7f29a00b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f29bc109190 tx=0x7f29a4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.398+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29ac009510 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.601+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f29bc02e640 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.609+0000 7f29b2a00640 1 -- 172.21.15.27:0/1678775992 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v35)=0 v35) ==== 121+0+0 (secure 0 0 0) 0x7f29ac0ade60 con 0x7f29bc103b60 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f29a00b6550 msgr2=0x7f29a00b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f29a00b6550 0x7f29a00b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f29bc109190 tx=0x7f29a4005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 msgr2=0x7f29bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 0x7f29bc075710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e9c0 tx=0x7f29ac00ee80 comp rx=0 tx=0).stop 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 shutdown_connections 2024-09-21T01:12:16.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f29a00b6550 0x7f29a00b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29bc109df0 0x7f29bc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29bc1094a0 0x7f29bc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 --2- 172.21.15.27:0/1678775992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc103b60 0x7f29bc075710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 >> 172.21.15.27:0/1678775992 conn(0x7f29bc07b850 msgr2=0x7f29bc107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 shutdown_connections 2024-09-21T01:12:16.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:16.611+0000 7f29c0e00640 1 -- 172.21.15.27:0/1678775992 wait complete. 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi027') 2024-09-21T01:12:16.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T01:12:17.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:17 smithi027 ceph-mon[30375]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:17.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:17 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' 2024-09-21T01:12:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:17 smithi098 ceph-mon[34372]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:17 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' 2024-09-21T01:12:17.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:17 smithi089 ceph-mon[34486]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:17 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' 2024-09-21T01:12:18.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 -- 172.21.15.27:0/867544272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 msgr2=0x7fd440112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 --2- 172.21.15.27:0/867544272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd440112670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd438009be0 tx=0x7fd43802a6c0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 -- 172.21.15.27:0/867544272 shutdown_connections 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 --2- 172.21.15.27:0/867544272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd440112670 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 --2- 172.21.15.27:0/867544272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd44010b4f0 0x7fd44010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 --2- 172.21.15.27:0/867544272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 0x7fd440105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.268+0000 7fd446600640 1 -- 172.21.15.27:0/867544272 >> 172.21.15.27:0/867544272 conn(0x7fd440101680 msgr2=0x7fd440103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.269+0000 7fd446600640 1 -- 172.21.15.27:0/867544272 shutdown_connections 2024-09-21T01:12:18.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.269+0000 7fd446600640 1 -- 172.21.15.27:0/867544272 wait complete. 2024-09-21T01:12:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 Processor -- start 2024-09-21T01:12:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 -- start start 2024-09-21T01:12:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 0x7fd44019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd44010b4f0 0x7fd44019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd4401a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd440115b00 con 0x7fd44010be40 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd440115980 con 0x7fd440105bb0 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd446600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd440115c80 con 0x7fd44010b4f0 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd445c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd4401a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd445c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd4401a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60796/0 (socket says 172.21.15.27:60796) 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd445c00640 1 -- 172.21.15.27:0/1996275215 learned_addr learned my addr 172.21.15.27:0/1996275215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.270+0000 7fd445200640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 0x7fd44019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd43fe00640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd44010b4f0 0x7fd44019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445c00640 1 -- 172.21.15.27:0/1996275215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd44010b4f0 msgr2=0x7fd44019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445c00640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd44010b4f0 0x7fd44019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445c00640 1 -- 172.21.15.27:0/1996275215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 msgr2=0x7fd44019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445c00640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 0x7fd44019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445c00640 1 -- 172.21.15.27:0/1996275215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4401a2420 con 0x7fd44010be40 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445200640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 0x7fd44019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.271+0000 7fd445c00640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd4401a1cf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd438009b30 tx=0x7fd438003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.272+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438040020 con 0x7fd44010be40 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.272+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd438030970 con 0x7fd44010be40 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.272+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4401a26b0 con 0x7fd44010be40 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.272+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438036040 con 0x7fd44010be40 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.272+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401a8e70 con 0x7fd44010be40 2024-09-21T01:12:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.274+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd440106e30 con 0x7fd44010be40 2024-09-21T01:12:18.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.277+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fd438043020 con 0x7fd44010be40 2024-09-21T01:12:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.278+0000 7fd43d600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd4280b64d0 0x7fd4280b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.279+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd438106770 con 0x7fd44010be40 2024-09-21T01:12:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.279+0000 7fd445200640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd4280b64d0 0x7fd4280b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.280+0000 7fd445200640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd4280b64d0 0x7fd4280b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd43400bfd0 tx=0x7fd43400b390 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:18.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.282+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd43804a020 con 0x7fd44010be40 2024-09-21T01:12:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 -- 172.21.15.27:0/262795589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d641092d0 msgr2=0x7f1d641096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 --2- 172.21.15.27:0/262795589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d641092d0 0x7f1d641096e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009900 tx=0x7f1d4c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 -- 172.21.15.27:0/262795589 shutdown_connections 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 --2- 172.21.15.27:0/262795589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d64109c20 0x7f1d64110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 --2- 172.21.15.27:0/262795589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d641092d0 0x7f1d641096e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 --2- 172.21.15.27:0/262795589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64103990 0x7f1d64103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 -- 172.21.15.27:0/262795589 >> 172.21.15.27:0/262795589 conn(0x7f1d640ff3e0 msgr2=0x7f1d64101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 -- 172.21.15.27:0/262795589 shutdown_connections 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.559+0000 7f1d63e00640 1 -- 172.21.15.27:0/262795589 wait complete. 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.561+0000 7f1d63e00640 1 Processor -- start 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.561+0000 7f1d63e00640 1 -- start start 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.561+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 0x7f1d6419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d641092d0 0x7f1d6419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64109c20 0x7f1d641a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64113930 con 0x7f1d64103990 2024-09-21T01:12:18.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641137b0 con 0x7f1d64109c20 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64113ab0 con 0x7f1d641092d0 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 0x7f1d6419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 0x7f1d6419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60774/0 (socket says 172.21.15.27:60774) 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 -- 172.21.15.27:0/1878853517 learned_addr learned my addr 172.21.15.27:0/1878853517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d5be00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d641092d0 0x7f1d6419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63400640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64109c20 0x7f1d641a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 -- 172.21.15.27:0/1878853517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d641092d0 msgr2=0x7f1d6419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d641092d0 0x7f1d6419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 -- 172.21.15.27:0/1878853517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64109c20 msgr2=0x7f1d641a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64109c20 0x7f1d641a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d62a00640 1 -- 172.21.15.27:0/1878853517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641a2470 con 0x7f1d64103990 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.562+0000 7f1d63400640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64109c20 0x7f1d641a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.563+0000 7f1d62a00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 0x7f1d6419c500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400ea60 tx=0x7f1d5400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.563+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400cde0 con 0x7f1d64103990 2024-09-21T01:12:18.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.563+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d540035f0 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.563+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54010690 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.563+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641a2760 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.563+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641a8ec0 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.564+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d64103da0 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.567+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f1d540037a0 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.568+0000 7f1d5b400640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1d480b64d0 0x7f1d480b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.568+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1d540e5d60 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.569+0000 7f1d5be00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1d480b64d0 0x7f1d480b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.569+0000 7f1d5be00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1d480b64d0 0x7f1d480b89e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c02a9e0 tx=0x7f1d4c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.570+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d54056710 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.775+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1d6419d630 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.779+0000 7f1d5b400640 1 -- 172.21.15.27:0/1878853517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v36)=0 v36) ==== 138+0+0 (secure 0 0 0) 0x7f1d540ae150 con 0x7f1d64103990 2024-09-21T01:12:18.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1d480b64d0 msgr2=0x7f1d480b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1d480b64d0 0x7f1d480b89e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c02a9e0 tx=0x7f1d4c005a20 comp rx=0 tx=0).stop 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 msgr2=0x7f1d6419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 0x7f1d6419c500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400ea60 tx=0x7f1d5400ef20 comp rx=0 tx=0).stop 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 shutdown_connections 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1d480b64d0 0x7f1d480b89e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d64109c20 0x7f1d641a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d641092d0 0x7f1d6419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 --2- 172.21.15.27:0/1878853517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d64103990 0x7f1d6419c500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 >> 172.21.15.27:0/1878853517 conn(0x7f1d640ff3e0 msgr2=0x7f1d64108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 shutdown_connections 2024-09-21T01:12:18.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:17.782+0000 7f1d63e00640 1 -- 172.21.15.27:0/1878853517 wait complete. 2024-09-21T01:12:18.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.484+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd440105fc0 con 0x7fd44010be40 2024-09-21T01:12:18.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.488+0000 7fd43d600640 1 -- 172.21.15.27:0/1996275215 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v37)=0 v37) ==== 121+0+0 (secure 0 0 0) 0x7fd440105fc0 con 0x7fd44010be40 2024-09-21T01:12:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.490+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd4280b64d0 msgr2=0x7fd4280b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.490+0000 7fd446600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd4280b64d0 0x7fd4280b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd43400bfd0 tx=0x7fd43400b390 comp rx=0 tx=0).stop 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.490+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 msgr2=0x7fd4401a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.490+0000 7fd446600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd4401a1cf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd438009b30 tx=0x7fd438003cc0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 shutdown_connections 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd4280b64d0 0x7fd4280b89e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd44010be40 0x7fd4401a1cf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd44010b4f0 0x7fd44019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 --2- 172.21.15.27:0/1996275215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd440105bb0 0x7fd44019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 >> 172.21.15.27:0/1996275215 conn(0x7fd440101680 msgr2=0x7fd44010ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 shutdown_connections 2024-09-21T01:12:18.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:18.491+0000 7fd446600640 1 -- 172.21.15.27:0/1996275215 wait complete. 2024-09-21T01:12:19.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:18 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1878853517' entity='client.admin' 2024-09-21T01:12:19.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:19.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:19.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:19.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:19.054 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:18 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1996275215' entity='client.admin' 2024-09-21T01:12:19.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:18 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1878853517' entity='client.admin' 2024-09-21T01:12:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:18 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1996275215' entity='client.admin' 2024-09-21T01:12:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:18 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1878853517' entity='client.admin' 2024-09-21T01:12:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:18 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1996275215' entity='client.admin' 2024-09-21T01:12:19.558 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.558 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.558 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.559 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi027') 2024-09-21T01:12:19.559 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'fe42ec90-77b5-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi027') 2024-09-21T01:12:20.052 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T01:12:20.103 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:19 smithi027 ceph-mon[30375]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:20.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:19 smithi098 ceph-mon[34372]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:20.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:19 smithi089 ceph-mon[34486]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:20.426 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 -- 172.21.15.27:0/3341477253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 msgr2=0x7fad7410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 --2- 172.21.15.27:0/3341477253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 0x7fad7410d860 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad60009900 tx=0x7fad6002a410 comp rx=0 tx=0).stop 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 -- 172.21.15.27:0/3341477253 shutdown_connections 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 --2- 172.21.15.27:0/3341477253 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad7410de20 0x7fad74110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 --2- 172.21.15.27:0/3341477253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 0x7fad7410d860 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 --2- 172.21.15.27:0/3341477253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad741049c0 0x7fad74104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.080+0000 7fad7c400640 1 -- 172.21.15.27:0/3341477253 >> 172.21.15.27:0/3341477253 conn(0x7fad74079730 msgr2=0x7fad74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.081+0000 7fad7c400640 1 -- 172.21.15.27:0/3341477253 shutdown_connections 2024-09-21T01:12:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.081+0000 7fad7c400640 1 -- 172.21.15.27:0/3341477253 wait complete. 2024-09-21T01:12:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.082+0000 7fad7c400640 1 Processor -- start 2024-09-21T01:12:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.082+0000 7fad7c400640 1 -- start start 2024-09-21T01:12:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7c400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 0x7fad7419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7c400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 0x7fad7419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad7410de20 0x7fad741a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7c400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad74113ad0 con 0x7fad74105310 2024-09-21T01:12:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fad74113950 con 0x7fad7410de20 2024-09-21T01:12:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7c400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad74113c50 con 0x7fad741049c0 2024-09-21T01:12:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad79a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 0x7fad7419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad79000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 0x7fad7419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad79a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 0x7fad7419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:47716/0 (socket says 172.21.15.27:47716) 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad79a00640 1 -- 172.21.15.27:0/2252240080 learned_addr learned my addr 172.21.15.27:0/2252240080 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.083+0000 7fad7a400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad7410de20 0x7fad741a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad79a00640 1 -- 172.21.15.27:0/2252240080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad7410de20 msgr2=0x7fad741a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad79a00640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad7410de20 0x7fad741a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad79a00640 1 -- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 msgr2=0x7fad7419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad79a00640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 0x7fad7419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad79a00640 1 -- 172.21.15.27:0/2252240080 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad741a2610 con 0x7fad741049c0 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad7a400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad7410de20 0x7fad741a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad79a00640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 0x7fad7419c6a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad6800e9c0 tx=0x7fad6800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:21.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6800cd80 con 0x7fad741049c0 2024-09-21T01:12:21.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad680035f0 con 0x7fad741049c0 2024-09-21T01:12:21.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad741a2900 con 0x7fad741049c0 2024-09-21T01:12:21.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.084+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68010690 con 0x7fad741049c0 2024-09-21T01:12:21.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.085+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad741a9060 con 0x7fad741049c0 2024-09-21T01:12:21.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.086+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7406c1e0 con 0x7fad741049c0 2024-09-21T01:12:21.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.088+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7fad680037a0 con 0x7fad741049c0 2024-09-21T01:12:21.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.089+0000 7fad72000640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fad5c0b64d0 0x7fad5c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:21.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.090+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fad680e5f20 con 0x7fad741049c0 2024-09-21T01:12:21.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.090+0000 7fad79000640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fad5c0b64d0 0x7fad5c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:21.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.090+0000 7fad79000640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fad5c0b64d0 0x7fad5c0b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fad6002aa00 tx=0x7fad60005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:21.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.092+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad680ae310 con 0x7fad741049c0 2024-09-21T01:12:21.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.292+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fad74104dd0 con 0x7fad741049c0 2024-09-21T01:12:21.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.293+0000 7fad72000640 1 -- 172.21.15.27:0/2252240080 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7fad74104dd0 con 0x7fad741049c0 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global dev auth_debug true 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_file true 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_journald false 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global basic log_to_stderr false 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_allow_pool_delete true 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T01:12:21.295 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_bug true 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_old_message true 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global dev ms_die_on_skipped_message false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T01:12:21.296 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_mon 20/20 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_ms 1/1 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced debug_paxos 20/20 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T01:12:21.297 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced client_mount_timeout 30 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_client 20/20 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_mgr 20/20 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced debug_ms 1/1 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T01:12:21.298 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bdev_debug_aio true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_bluefs 20/20 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_bluestore 20/20 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_ms 1/1 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_osd 20/20 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_op_order true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_shutdown true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T01:12:21.299 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T01:12:21.300 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_mds 20/20 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced debug_ms 2/2 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T01:12:21.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_bal_split_size 100 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_debug_frag true 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_debug_scatterstat true 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds dev mds_verify_scatter true 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced client_mount_timeout 600 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced debug_client 20/20 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced debug_ms 1/1 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T01:12:21.302 INFO:teuthology.orchestra.run.smithi027.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T01:12:21.303 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T01:12:21.303 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T01:12:21.303 INFO:teuthology.orchestra.run.smithi027.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T01:12:21.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.297+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fad5c0b64d0 msgr2=0x7fad5c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:21.303 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.297+0000 7fad7c400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fad5c0b64d0 0x7fad5c0b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fad6002aa00 tx=0x7fad60005a20 comp rx=0 tx=0).stop 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.297+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 msgr2=0x7fad7419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.297+0000 7fad7c400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 0x7fad7419c6a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fad6800e9c0 tx=0x7fad6800ee80 comp rx=0 tx=0).stop 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.297+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 shutdown_connections 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.297+0000 7fad7c400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fad5c0b64d0 0x7fad5c0b89e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.298+0000 7fad7c400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fad7410de20 0x7fad741a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.298+0000 7fad7c400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad74105310 0x7fad7419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.298+0000 7fad7c400640 1 --2- 172.21.15.27:0/2252240080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad741049c0 0x7fad7419c6a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.298+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 >> 172.21.15.27:0/2252240080 conn(0x7fad74079730 msgr2=0x7fad7410ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.298+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 shutdown_connections 2024-09-21T01:12:21.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:21.298+0000 7fad7c400640 1 -- 172.21.15.27:0/2252240080 wait complete. 2024-09-21T01:12:21.976 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:21 smithi027 ceph-mon[30375]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:21.977 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2252240080' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:12:22.016 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T01:12:22.017 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:12:22.017 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:12:22.053 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:12:22.053 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 01:10:49.952076484 +0000 2024-09-21T01:12:22.130 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 01:06:04.014195896 +0000 2024-09-21T01:12:22.131 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 01:06:04.014195896 +0000 2024-09-21T01:12:22.131 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 01:06:04.014195896 +0000 2024-09-21T01:12:22.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:12:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:21 smithi098 ceph-mon[34372]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2252240080' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:12:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T01:12:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T01:12:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000287031 s, 1.8 MB/s 2024-09-21T01:12:22.228 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:12:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:21 smithi089 ceph-mon[34486]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2252240080' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T01:12:22.266 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-09-21T01:12:22.342 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:12:22.342 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:22.342 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-21T01:12:22.342 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:22.342 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:22.343 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 01:10:49.991075495 +0000 2024-09-21T01:12:22.343 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 01:06:04.482184082 +0000 2024-09-21T01:12:22.343 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 01:06:04.482184082 +0000 2024-09-21T01:12:22.343 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 01:06:04.482184082 +0000 2024-09-21T01:12:22.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:12:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T01:12:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T01:12:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000291687 s, 1.8 MB/s 2024-09-21T01:12:22.439 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:12:22.476 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 01:10:50.036074355 +0000 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 01:06:04.930172773 +0000 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 01:06:04.930172773 +0000 2024-09-21T01:12:22.552 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 01:06:04.930172773 +0000 2024-09-21T01:12:22.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:12:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T01:12:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T01:12:22.646 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00025056 s, 2.0 MB/s 2024-09-21T01:12:22.649 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:12:22.686 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-09-21T01:12:22.762 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:12:22.762 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:22.762 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-21T01:12:22.762 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:22.762 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:22.763 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 01:10:50.074073392 +0000 2024-09-21T01:12:22.763 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 01:06:05.398160959 +0000 2024-09-21T01:12:22.763 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 01:06:05.398160959 +0000 2024-09-21T01:12:22.763 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 01:06:05.398160959 +0000 2024-09-21T01:12:22.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:12:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-09-21T01:12:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-09-21T01:12:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00027871 s, 1.8 MB/s 2024-09-21T01:12:22.859 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:12:22.896 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-09-21T01:12:22.896 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:12:22.915 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:12:22.915 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-09-21T01:12:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:12:22.974 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:22.974 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-21T01:12:22.974 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:22.974 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:22.975 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 01:11:53.955270036 +0000 2024-09-21T01:12:22.975 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 01:05:27.930214684 +0000 2024-09-21T01:12:22.975 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 01:05:27.930214684 +0000 2024-09-21T01:12:22.975 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 01:05:27.930214684 +0000 2024-09-21T01:12:22.975 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:12:23.044 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T01:12:23.045 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T01:12:23.045 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000270762 s, 1.9 MB/s 2024-09-21T01:12:23.046 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:12:23.104 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 01:11:53.955270036 +0000 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 01:05:28.263206970 +0000 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 01:05:28.263206970 +0000 2024-09-21T01:12:23.162 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 01:05:28.263206970 +0000 2024-09-21T01:12:23.163 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:12:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T01:12:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T01:12:23.227 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000319533 s, 1.6 MB/s 2024-09-21T01:12:23.229 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:12:23.286 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 01:11:53.956270013 +0000 2024-09-21T01:12:23.342 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 01:05:28.583199604 +0000 2024-09-21T01:12:23.343 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 01:05:28.583199604 +0000 2024-09-21T01:12:23.343 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 01:05:28.583199604 +0000 2024-09-21T01:12:23.343 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:12:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T01:12:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T01:12:23.406 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000198451 s, 2.6 MB/s 2024-09-21T01:12:23.407 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:12:23.464 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-09-21 01:11:53.956270013 +0000 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-09-21 01:05:28.906192215 +0000 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-09-21 01:05:28.906192215 +0000 2024-09-21T01:12:23.521 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-09-21 01:05:28.906192215 +0000 2024-09-21T01:12:23.521 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:12:23.584 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-09-21T01:12:23.584 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-09-21T01:12:23.584 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000264709 s, 1.9 MB/s 2024-09-21T01:12:23.586 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:12:23.642 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-21T01:12:23.642 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T01:12:23.662 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T01:12:23.662 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-09-21T01:12:23.720 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T01:12:23.720 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:23.720 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-21T01:12:23.720 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 01:11:51.560108127 +0000 2024-09-21T01:12:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 01:05:37.694368106 +0000 2024-09-21T01:12:23.721 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 01:05:37.694368106 +0000 2024-09-21T01:12:23.721 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 01:05:37.694368106 +0000 2024-09-21T01:12:23.721 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T01:12:23.789 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T01:12:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T01:12:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000285703 s, 1.8 MB/s 2024-09-21T01:12:23.791 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T01:12:23.850 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-09-21T01:12:23.906 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T01:12:23.906 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:23.906 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T01:12:23.907 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:23.907 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:23.907 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 01:11:51.560108127 +0000 2024-09-21T01:12:23.907 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 01:05:38.025363891 +0000 2024-09-21T01:12:23.907 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 01:05:38.025363891 +0000 2024-09-21T01:12:23.907 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 01:05:38.025363891 +0000 2024-09-21T01:12:23.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T01:12:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T01:12:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T01:12:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000212609 s, 2.4 MB/s 2024-09-21T01:12:23.973 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T01:12:24.030 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-09-21T01:12:24.086 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T01:12:24.086 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:24.086 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T01:12:24.087 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:24.087 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:24.087 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 01:11:51.561108115 +0000 2024-09-21T01:12:24.087 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 01:05:38.354359701 +0000 2024-09-21T01:12:24.087 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 01:05:38.354359701 +0000 2024-09-21T01:12:24.087 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 01:05:38.354359701 +0000 2024-09-21T01:12:24.087 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T01:12:24.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:23 smithi098 ceph-mon[34372]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:24.151 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T01:12:24.151 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T01:12:24.151 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000199833 s, 2.6 MB/s 2024-09-21T01:12:24.152 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T01:12:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:23 smithi027 ceph-mon[30375]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:24.209 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-09-21T01:12:24.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:23 smithi089 ceph-mon[34486]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-21 01:11:51.561108115 +0000 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-21 01:05:38.675355613 +0000 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-21 01:05:38.675355613 +0000 2024-09-21T01:12:24.265 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-09-21 01:05:38.675355613 +0000 2024-09-21T01:12:24.266 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T01:12:24.328 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-21T01:12:24.328 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-21T01:12:24.328 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000206045 s, 2.5 MB/s 2024-09-21T01:12:24.330 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T01:12:24.386 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2024-09-21T01:12:24.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:12:24.766 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:12:25.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:25 smithi027 ceph-mon[30375]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:25 smithi098 ceph-mon[34372]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:26.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:25 smithi089 ceph-mon[34486]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:27.623 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:12:27.655 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-09-21T01:12:27.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:27 smithi027 ceph-mon[30375]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:28.028 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:12:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:27 smithi098 ceph-mon[34372]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:27 smithi089 ceph-mon[34486]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 -- 172.21.15.27:0/2337749066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec81092d0 msgr2=0x7f1ec81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2337749066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec81092d0 0x7f1ec81096e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009900 tx=0x7f1eb402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 -- 172.21.15.27:0/2337749066 shutdown_connections 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2337749066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8109c20 0x7f1ec8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2337749066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec81092d0 0x7f1ec81096e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2337749066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec8103990 0x7f1ec8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 -- 172.21.15.27:0/2337749066 >> 172.21.15.27:0/2337749066 conn(0x7f1ec80ff3e0 msgr2=0x7f1ec8101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 -- 172.21.15.27:0/2337749066 shutdown_connections 2024-09-21T01:12:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.622+0000 7f1ecce00640 1 -- 172.21.15.27:0/2337749066 wait complete. 2024-09-21T01:12:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.623+0000 7f1ecce00640 1 Processor -- start 2024-09-21T01:12:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.623+0000 7f1ecce00640 1 -- start start 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ecce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8103990 0x7f1ec819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ecce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 0x7f1ec819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ecce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8109c20 0x7f1ec819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ecce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81138e0 con 0x7f1ec8103990 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ecce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8113760 con 0x7f1ec8109c20 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ecce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8113a60 con 0x7f1ec81092d0 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 0x7f1ec819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8103990 0x7f1ec819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 0x7f1ec819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51402/0 (socket says 172.21.15.27:51402) 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 -- 172.21.15.27:0/2570446729 learned_addr learned my addr 172.21.15.27:0/2570446729 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:12:28.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec7e00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8109c20 0x7f1ec819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 -- 172.21.15.27:0/2570446729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8109c20 msgr2=0x7f1ec819fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8109c20 0x7f1ec819fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 -- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8103990 msgr2=0x7f1ec819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8103990 0x7f1ec819a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec6a00640 1 -- 172.21.15.27:0/2570446729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a02c0 con 0x7f1ec81092d0 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.624+0000 7f1ec7400640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8103990 0x7f1ec819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.625+0000 7f1ec6a00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 0x7f1ec819a8f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1eb402a910 tx=0x7f1eb4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.625+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb403f030 con 0x7f1ec81092d0 2024-09-21T01:12:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.625+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb40043d0 con 0x7f1ec81092d0 2024-09-21T01:12:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.625+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a0550 con 0x7f1ec81092d0 2024-09-21T01:12:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.625+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4035070 con 0x7f1ec81092d0 2024-09-21T01:12:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.625+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a6d10 con 0x7f1ec81092d0 2024-09-21T01:12:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.627+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec8104c10 con 0x7f1ec81092d0 2024-09-21T01:12:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.629+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f1eb4042030 con 0x7f1ec81092d0 2024-09-21T01:12:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.630+0000 7f1ebfe00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eac0b64d0 0x7f1eac0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.630+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1eb4105ad0 con 0x7f1ec81092d0 2024-09-21T01:12:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.631+0000 7f1ec7400640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eac0b64d0 0x7f1eac0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:28.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.631+0000 7f1ec7400640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eac0b64d0 0x7f1eac0b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fc0 tx=0x7f1eb8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.635+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1eb4107e20 con 0x7f1ec81092d0 2024-09-21T01:12:28.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:28.842+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1ec8063800 con 0x7f1eac0b64d0 2024-09-21T01:12:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:29 smithi098 ceph-mon[34372]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:29 smithi098 ceph-mon[34372]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:12:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:29 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:12:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:29 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:12:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:29 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:29 smithi027 ceph-mon[30375]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:29 smithi027 ceph-mon[30375]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:12:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:29 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:12:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:29 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:12:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:29 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:29 smithi089 ceph-mon[34486]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:29 smithi089 ceph-mon[34486]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:12:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:29 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:12:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:29 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:12:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:29 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:31 smithi098 ceph-mon[34372]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:31 smithi027 ceph-mon[30375]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:32.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:31 smithi089 ceph-mon[34486]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:33.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:32 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/943984861' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac7acdbc-3b48-4cdc-b04e-46d6930e274d"} : dispatch 2024-09-21T01:12:33.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:32 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/943984861' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac7acdbc-3b48-4cdc-b04e-46d6930e274d"}]': finished 2024-09-21T01:12:33.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:32 smithi098 ceph-mon[34372]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:12:33.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:32 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:33.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:32 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/943984861' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac7acdbc-3b48-4cdc-b04e-46d6930e274d"} : dispatch 2024-09-21T01:12:33.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:32 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/943984861' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac7acdbc-3b48-4cdc-b04e-46d6930e274d"}]': finished 2024-09-21T01:12:33.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:32 smithi027 ceph-mon[30375]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:12:33.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:32 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:33.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:32 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/943984861' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ac7acdbc-3b48-4cdc-b04e-46d6930e274d"} : dispatch 2024-09-21T01:12:33.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:32 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/943984861' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac7acdbc-3b48-4cdc-b04e-46d6930e274d"}]': finished 2024-09-21T01:12:33.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:32 smithi089 ceph-mon[34486]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T01:12:33.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:32 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:33 smithi098 ceph-mon[34372]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:33 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1665832045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:12:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:33 smithi027 ceph-mon[30375]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:33 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1665832045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:12:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:33 smithi089 ceph-mon[34486]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:33 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1665832045' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:12:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:35 smithi098 ceph-mon[34372]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:35 smithi027 ceph-mon[30375]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:36.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:35 smithi089 ceph-mon[34486]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:38.142 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:37 smithi027 ceph-mon[30375]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:38.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:37 smithi098 ceph-mon[34372]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:38.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:37 smithi089 ceph-mon[34486]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:39.948 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:39 smithi027 ceph-mon[30375]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:40.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:39 smithi098 ceph-mon[34372]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:40.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:39 smithi089 ceph-mon[34486]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:42.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:41 smithi027 ceph-mon[30375]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:42.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:12:42.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:41 smithi098 ceph-mon[34372]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:12:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:41 smithi089 ceph-mon[34486]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T01:12:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:43.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:42 smithi098 ceph-mon[34372]: Deploying daemon osd.0 on smithi027 2024-09-21T01:12:43.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:42 smithi027 ceph-mon[30375]: Deploying daemon osd.0 on smithi027 2024-09-21T01:12:43.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:42 smithi089 ceph-mon[34486]: Deploying daemon osd.0 on smithi027 2024-09-21T01:12:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:43 smithi098 ceph-mon[34372]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:43 smithi027 ceph-mon[30375]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:44.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:43 smithi089 ceph-mon[34486]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:46.073 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:45 smithi027 ceph-mon[30375]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:46.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:45 smithi098 ceph-mon[34372]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:46.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:45 smithi089 ceph-mon[34486]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:47 smithi098 ceph-mon[34372]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:47 smithi027 ceph-mon[30375]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:48.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:47 smithi089 ceph-mon[34486]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:49.899 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:49 smithi027 ceph-mon[30375]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:49 smithi098 ceph-mon[34372]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:50.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:49 smithi089 ceph-mon[34486]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:50.904 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:50.904 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:50.905 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:51.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:51.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:51.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:51.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:51.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:51.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:52 smithi027 ceph-mon[30375]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:52.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:52 smithi098 ceph-mon[34372]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:52.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:52 smithi089 ceph-mon[34486]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:53 smithi098 ceph-mon[34372]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:53 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:53 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:53 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:53 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:53 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:53 smithi027 ceph-mon[30375]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:53 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:53 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:53 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:53 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:53 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:53.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:53 smithi089 ceph-mon[34486]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:53 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:53 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:53 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:12:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:53 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:12:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:53 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:55.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.722+0000 7f1ebfe00640 1 -- 172.21.15.27:0/2570446729 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ec8063800 con 0x7f1eac0b64d0 2024-09-21T01:12:55.725 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eac0b64d0 msgr2=0x7f1eac0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eac0b64d0 0x7f1eac0b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009fc0 tx=0x7f1eb8005f90 comp rx=0 tx=0).stop 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 msgr2=0x7f1ec819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 0x7f1ec819a8f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1eb402a910 tx=0x7f1eb4031a20 comp rx=0 tx=0).stop 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 shutdown_connections 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eac0b64d0 0x7f1eac0b89e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8109c20 0x7f1ec819fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec81092d0 0x7f1ec819a8f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 --2- 172.21.15.27:0/2570446729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8103990 0x7f1ec819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 >> 172.21.15.27:0/2570446729 conn(0x7f1ec80ff3e0 msgr2=0x7f1ec8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 shutdown_connections 2024-09-21T01:12:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:12:55.725+0000 7f1ecce00640 1 -- 172.21.15.27:0/2570446729 wait complete. 2024-09-21T01:12:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:55 smithi098 ceph-mon[34372]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:55.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:55 smithi027 ceph-mon[30375]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:55 smithi089 ceph-mon[34486]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:56.450 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T01:12:56.453 INFO:tasks.cephadm:Deploying osd.1 on smithi089 with /dev/vg_nvme/lv_4... 2024-09-21T01:12:56.454 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:12:56.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:56.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:56.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:56.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:56 smithi027 ceph-mon[30375]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:12:56.702 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:12:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:56 smithi089 ceph-mon[34486]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:12:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:12:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:12:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:56 smithi098 ceph-mon[34372]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T01:12:57.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-mon[30375]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:57.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-mon[30375]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:12:57.757 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-mon[30375]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:12:57.758 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-mon[30375]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:12:57.758 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:57.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:57 smithi089 ceph-mon[34486]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:57.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:57 smithi089 ceph-mon[34486]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:12:57.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:57 smithi089 ceph-mon[34486]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:12:57.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:57 smithi089 ceph-mon[34486]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:12:57.997 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:57 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:58.074 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0[47260]: 2024-09-21T01:12:57.756+0000 7f394aa00640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T01:12:58.074 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:12:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0[47260]: 2024-09-21T01:12:57.760+0000 7f3945a00640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:12:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:57 smithi098 ceph-mon[34372]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:57 smithi098 ceph-mon[34372]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T01:12:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:57 smithi098 ceph-mon[34372]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T01:12:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:57 smithi098 ceph-mon[34372]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:12:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:57 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:58.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:58 smithi089 ceph-mon[34486]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:12:58.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:58 smithi089 ceph-mon[34486]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T01:12:58.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:58.910 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:59.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:58 smithi098 ceph-mon[34372]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:12:59.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:58 smithi098 ceph-mon[34372]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T01:12:59.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:59.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:59.180 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:12:59.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:58 smithi027 ceph-mon[30375]: from='osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:12:59.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:58 smithi027 ceph-mon[30375]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T01:12:59.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:59.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:59.207 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2024-09-21T01:12:59.455 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:12:59.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:59 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:12:59.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:59 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:12:59.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:59 smithi027 ceph-mon[30375]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:12:59.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:59 smithi027 ceph-mon[30375]: osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] boot 2024-09-21T01:12:59.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:59 smithi027 ceph-mon[30375]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:12:59.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:12:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:12:59.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 -- 172.21.15.89:0/3512306048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 msgr2=0x7f40fc105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:59.959 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3512306048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc105580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009900 tx=0x7f40e402a410 comp rx=0 tx=0).stop 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 -- 172.21.15.89:0/3512306048 shutdown_connections 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3512306048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 0x7f40fc10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3512306048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc105580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3512306048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40fc1031a0 0x7f40fc1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 -- 172.21.15.89:0/3512306048 >> 172.21.15.89:0/3512306048 conn(0x7f40fc077c60 msgr2=0x7f40fc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 -- 172.21.15.89:0/3512306048 shutdown_connections 2024-09-21T01:12:59.960 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.958+0000 7f4101b6d640 1 -- 172.21.15.89:0/3512306048 wait complete. 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 Processor -- start 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 -- start start 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40fc1031a0 0x7f40fc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 0x7f40fc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1129c0 con 0x7f40fc105ac0 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc112840 con 0x7f40fc105170 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f4101b6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc112b40 con 0x7f40fc1031a0 2024-09-21T01:12:59.962 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f40faffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f40faffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36196/0 (socket says 172.21.15.89:36196) 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f40fb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40fc1031a0 0x7f40fc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.959+0000 7f40faffd640 1 -- 172.21.15.89:0/3200933082 learned_addr learned my addr 172.21.15.89:0/3200933082 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40fbfff640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 0x7f40fc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40faffd640 1 -- 172.21.15.89:0/3200933082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40fc1031a0 msgr2=0x7f40fc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40faffd640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40fc1031a0 0x7f40fc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40faffd640 1 -- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 msgr2=0x7f40fc19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40faffd640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 0x7f40fc19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40faffd640 1 -- 172.21.15.89:0/3200933082 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc1a0450 con 0x7f40fc105170 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40fbfff640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 0x7f40fc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40faffd640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc19aa80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40e402a930 tx=0x7f40e4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e403f030 con 0x7f40fc105170 2024-09-21T01:12:59.963 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e4003aa0 con 0x7f40fc105170 2024-09-21T01:12:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc1a06e0 con 0x7f40fc105170 2024-09-21T01:12:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4035070 con 0x7f40fc105170 2024-09-21T01:12:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.960+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc1a6ea0 con 0x7f40fc105170 2024-09-21T01:12:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.962+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40fc06ac30 con 0x7f40fc105170 2024-09-21T01:12:59.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.963+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f40e4042030 con 0x7f40fc105170 2024-09-21T01:12:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.964+0000 7f40f8ff9640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f40e80b64d0 0x7f40e80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:12:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.964+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7f40e4106420 con 0x7f40fc105170 2024-09-21T01:12:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.964+0000 7f40fb7fe640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f40e80b64d0 0x7f40e80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:12:59.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.964+0000 7f40fb7fe640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f40e80b64d0 0x7f40e80b89e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005fd0 tx=0x7f40ec009680 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:12:59.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:12:59.967+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40e40ce650 con 0x7f40fc105170 2024-09-21T01:12:59.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:59 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:13:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:59 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:13:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:59 smithi089 ceph-mon[34486]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:13:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:59 smithi089 ceph-mon[34486]: osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] boot 2024-09-21T01:13:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:59 smithi089 ceph-mon[34486]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:13:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:12:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:13:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:59 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:13:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:59 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:13:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:59 smithi098 ceph-mon[34372]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T01:13:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:59 smithi098 ceph-mon[34372]: osd.0 [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] boot 2024-09-21T01:13:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:59 smithi098 ceph-mon[34372]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T01:13:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:12:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T01:13:00.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:00.163+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f40fc063800 con 0x7f40e80b64d0 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:01.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:01.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:01.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:01.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:01.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:01.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:01.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:01.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:01.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:01.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:01 smithi089 ceph-mon[34486]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:01.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:01 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi027 2024-09-21T01:13:01.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:01 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T01:13:01.781 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:01 smithi089 ceph-mon[34486]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:01 smithi098 ceph-mon[34372]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:01 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi027 2024-09-21T01:13:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:01 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T01:13:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:01 smithi098 ceph-mon[34372]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:01 smithi027 ceph-mon[30375]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:01 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi027 2024-09-21T01:13:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:01 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi027 to 17089M 2024-09-21T01:13:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:01 smithi027 ceph-mon[30375]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:02 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/780090203' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"} : dispatch 2024-09-21T01:13:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:02 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"} : dispatch 2024-09-21T01:13:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:02 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"}]': finished 2024-09-21T01:13:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:02 smithi089 ceph-mon[34486]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:13:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:02 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:02 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/780090203' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"} : dispatch 2024-09-21T01:13:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:02 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"} : dispatch 2024-09-21T01:13:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:02 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"}]': finished 2024-09-21T01:13:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:02 smithi098 ceph-mon[34372]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:13:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:02 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:02 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/780090203' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"} : dispatch 2024-09-21T01:13:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:02 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"} : dispatch 2024-09-21T01:13:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:02 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5714e04c-89ef-47b5-9d28-8cc23530a45e"}]': finished 2024-09-21T01:13:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:02 smithi027 ceph-mon[30375]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T01:13:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:02 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:03 smithi098 ceph-mon[34372]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:03 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/651747008' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:03 smithi027 ceph-mon[30375]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:03 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/651747008' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:03 smithi089 ceph-mon[34486]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:03 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/651747008' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:05 smithi098 ceph-mon[34372]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:05 smithi027 ceph-mon[30375]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:05 smithi089 ceph-mon[34486]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:08.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:07 smithi089 ceph-mon[34486]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:07 smithi098 ceph-mon[34372]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:07 smithi027 ceph-mon[30375]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:09.954 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:09 smithi089 ceph-mon[34486]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:09 smithi098 ceph-mon[34372]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:09 smithi027 ceph-mon[30375]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:11.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:11 smithi089 ceph-mon[34486]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:13:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:11 smithi089 ceph-mon[34486]: Deploying daemon osd.1 on smithi089 2024-09-21T01:13:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:11 smithi098 ceph-mon[34372]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:13:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:11 smithi098 ceph-mon[34372]: Deploying daemon osd.1 on smithi089 2024-09-21T01:13:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:11 smithi027 ceph-mon[30375]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T01:13:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:11 smithi027 ceph-mon[30375]: Deploying daemon osd.1 on smithi089 2024-09-21T01:13:14.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:13 smithi089 ceph-mon[34486]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:14.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:13 smithi098 ceph-mon[34372]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:14.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:13 smithi027 ceph-mon[30375]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:15 smithi098 ceph-mon[34372]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:15 smithi027 ceph-mon[30375]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:16.231 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:15 smithi089 ceph-mon[34486]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:18.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:17 smithi089 ceph-mon[34486]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:18.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:18.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:18.054 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:17 smithi098 ceph-mon[34372]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:17 smithi027 ceph-mon[30375]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:19 smithi098 ceph-mon[34372]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:19 smithi027 ceph-mon[30375]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:20.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:19 smithi089 ceph-mon[34486]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:20.969 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 1 on host 'smithi089' 2024-09-21T01:13:20.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.968+0000 7f40f8ff9640 1 -- 172.21.15.89:0/3200933082 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f40fc063800 con 0x7f40e80b64d0 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f40e80b64d0 msgr2=0x7f40e80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f40e80b64d0 0x7f40e80b89e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005fd0 tx=0x7f40ec009680 comp rx=0 tx=0).stop 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 msgr2=0x7f40fc19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc19aa80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40e402a930 tx=0x7f40e4004270 comp rx=0 tx=0).stop 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 shutdown_connections 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f40e80b64d0 0x7f40e80b89e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:20.971 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40fc105ac0 0x7f40fc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:20.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40fc105170 0x7f40fc19aa80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:20.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 --2- 172.21.15.89:0/3200933082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40fc1031a0 0x7f40fc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:20.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 >> 172.21.15.89:0/3200933082 conn(0x7f40fc077c60 msgr2=0x7f40fc101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:13:20.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 shutdown_connections 2024-09-21T01:13:20.972 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:13:20.971+0000 7f4101b6d640 1 -- 172.21.15.89:0/3200933082 wait complete. 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:21.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:21.605 DEBUG:teuthology.orchestra.run.smithi089:osd.1> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T01:13:21.607 INFO:tasks.cephadm:Deploying osd.2 on smithi098 with /dev/vg_nvme/lv_4... 2024-09-21T01:13:21.607 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T01:13:21.857 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:13:22.244 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:22 smithi098 ceph-mon[34372]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:22.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:22 smithi089 ceph-mon[34486]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:22 smithi027 ceph-mon[30375]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:23.750 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1[39082]: 2024-09-21T01:13:23.448+0000 7fc0f0603640 -1 osd.1 0 log_to_monitors true 2024-09-21T01:13:24.200 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:13:24.232 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2024-09-21T01:13:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:24 smithi089 ceph-mon[34486]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:24 smithi089 ceph-mon[34486]: from='osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:13:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:24 smithi089 ceph-mon[34486]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:13:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:24 smithi098 ceph-mon[34372]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:24 smithi098 ceph-mon[34372]: from='osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:13:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:24 smithi098 ceph-mon[34372]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:13:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:24 smithi027 ceph-mon[30375]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:24 smithi027 ceph-mon[30375]: from='osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:13:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:24 smithi027 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T01:13:24.484 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:13:24.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 -- 172.21.15.98:0/2185754951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370108cf0 msgr2=0x7f737010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 --2- 172.21.15.98:0/2185754951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370108cf0 0x7f737010f520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7368009ca0 tx=0x7f736802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 -- 172.21.15.98:0/2185754951 shutdown_connections 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 --2- 172.21.15.98:0/2185754951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370108cf0 0x7f737010f520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 --2- 172.21.15.98:0/2185754951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73701083a0 0x7f73701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 --2- 172.21.15.98:0/2185754951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370102a60 0x7f7370102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 -- 172.21.15.98:0/2185754951 >> 172.21.15.98:0/2185754951 conn(0x7f7370079730 msgr2=0x7f7370079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 -- 172.21.15.98:0/2185754951 shutdown_connections 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.918+0000 7f73756c9640 1 -- 172.21.15.98:0/2185754951 wait complete. 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 Processor -- start 2024-09-21T01:13:24.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 -- start start 2024-09-21T01:13:24.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 0x7f73700ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:24.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73701083a0 0x7f73700ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:24.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370108cf0 0x7f73701000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:24.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7370112960 con 0x7f73701083a0 2024-09-21T01:13:24.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f73701127e0 con 0x7f7370108cf0 2024-09-21T01:13:24.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f73756c9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7370112ae0 con 0x7f7370102a60 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f736effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 0x7f73700ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f736effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 0x7f73700ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34328/0 (socket says 172.21.15.98:34328) 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f736effd640 1 -- 172.21.15.98:0/2241547066 learned_addr learned my addr 172.21.15.98:0/2241547066 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736f7fe640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370108cf0 0x7f73701000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.919+0000 7f736e7fc640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73701083a0 0x7f73700ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736effd640 1 -- 172.21.15.98:0/2241547066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370108cf0 msgr2=0x7f73701000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736effd640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370108cf0 0x7f73701000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736effd640 1 -- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73701083a0 msgr2=0x7f73700ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736effd640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73701083a0 0x7f73700ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736effd640 1 -- 172.21.15.98:0/2241547066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73701a86e0 con 0x7f7370102a60 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736f7fe640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370108cf0 0x7f73701000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f736effd640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 0x7f73700ff670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f736000c970 tx=0x7f736000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360007bb0 con 0x7f7370102a60 2024-09-21T01:13:24.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360007d50 con 0x7f7370102a60 2024-09-21T01:13:24.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73701a89d0 con 0x7f7370102a60 2024-09-21T01:13:24.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000f450 con 0x7f7370102a60 2024-09-21T01:13:24.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.920+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701a8ec0 con 0x7f7370102a60 2024-09-21T01:13:24.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.921+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7370102e70 con 0x7f7370102a60 2024-09-21T01:13:24.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.924+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f7360004f00 con 0x7f7370102a60 2024-09-21T01:13:24.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.924+0000 7f7357fff640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73580b64d0 0x7f73580b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:24.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.925+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2118+0+0 (secure 0 0 0) 0x7f73600e5bc0 con 0x7f7370102a60 2024-09-21T01:13:24.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.925+0000 7f736e7fc640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73580b64d0 0x7f73580b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:24.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.925+0000 7f736e7fc640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73580b64d0 0x7f73580b89e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f737006a380 tx=0x7f736400a4c0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:13:24.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:24.928+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7360014030 con 0x7f7370102a60 2024-09-21T01:13:25.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:25.119+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7370063800 con 0x7f73580b64d0 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.122 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:25.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:25.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:25 smithi098 ceph-mon[34372]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:13:25.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:13:25.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:13:25.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:13:25.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:25 smithi027 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:13:25.500 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1[39082]: 2024-09-21T01:13:25.037+0000 7fc0ec585640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T01:13:25.500 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1[39082]: 2024-09-21T01:13:25.041+0000 7fc0e8383640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:25.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:25.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:25.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:25.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:25 smithi089 ceph-mon[34486]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi089 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi089 2024-09-21T01:13:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T01:13:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T01:13:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:26.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:26.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:26.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:26.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi089 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi089 to 17087M 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:27 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:13:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:27 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:13:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:27 smithi098 ceph-mon[34372]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:27 smithi098 ceph-mon[34372]: osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] boot 2024-09-21T01:13:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:27 smithi098 ceph-mon[34372]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T01:13:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:27 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:27 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:13:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:27 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:13:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:27 smithi027 ceph-mon[30375]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:27 smithi027 ceph-mon[30375]: osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] boot 2024-09-21T01:13:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:27 smithi027 ceph-mon[30375]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T01:13:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:27 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:27 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:13:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:27 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:13:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:27 smithi089 ceph-mon[34486]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:27 smithi089 ceph-mon[34486]: osd.1 [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] boot 2024-09-21T01:13:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:27 smithi089 ceph-mon[34486]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T01:13:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:27 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T01:13:28.332 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/115614461' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"} : dispatch 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"} : dispatch 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"}]': finished 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:28.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:28 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/4095499861' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/115614461' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"} : dispatch 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"} : dispatch 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"}]': finished 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:28 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/4095499861' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/115614461' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"} : dispatch 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"} : dispatch 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3493a644-896e-4c12-bcf2-62a3aaa3f7d2"}]': finished 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:28.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:28 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/4095499861' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:30 smithi027 ceph-mon[30375]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:30.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:30 smithi098 ceph-mon[34372]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:30.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:30 smithi089 ceph-mon[34486]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:32 smithi098 ceph-mon[34372]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:32.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:32 smithi027 ceph-mon[30375]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:32 smithi089 ceph-mon[34486]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:34.388 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:34 smithi098 ceph-mon[34372]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:34 smithi027 ceph-mon[30375]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:34 smithi089 ceph-mon[34486]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:36.332 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:36 smithi098 ceph-mon[34372]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:36.347 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:36 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:13:36.347 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:36 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:36 smithi027 ceph-mon[30375]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:36 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:13:36.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:36 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:36 smithi089 ceph-mon[34486]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:36 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T01:13:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:36 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:37.220 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:37 smithi098 ceph-mon[34372]: Deploying daemon osd.2 on smithi098 2024-09-21T01:13:37.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:37 smithi027 ceph-mon[30375]: Deploying daemon osd.2 on smithi098 2024-09-21T01:13:37.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:37 smithi089 ceph-mon[34486]: Deploying daemon osd.2 on smithi098 2024-09-21T01:13:38.387 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:38 smithi098 ceph-mon[34372]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:38 smithi027 ceph-mon[30375]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:38 smithi089 ceph-mon[34486]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:40 smithi098 ceph-mon[34372]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:40.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:40 smithi027 ceph-mon[30375]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:40.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:40 smithi089 ceph-mon[34486]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:42.341 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:42 smithi098 ceph-mon[34372]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:42.342 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:42.342 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:42.342 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:42.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:42 smithi027 ceph-mon[30375]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:42.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:42.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:42.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:42 smithi089 ceph-mon[34486]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:44 smithi098 ceph-mon[34372]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:44.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:44 smithi027 ceph-mon[30375]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:44 smithi089 ceph-mon[34486]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:45.845 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:45 smithi098 ceph-mon[34372]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:45.846 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:45.846 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:45.846 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:45.846 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:45.846 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:45 smithi027 ceph-mon[30375]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:45 smithi089 ceph-mon[34486]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.181+0000 7f7357fff640 1 -- 172.21.15.98:0/2241547066 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7370063800 con 0x7f73580b64d0 2024-09-21T01:13:46.181 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 2 on host 'smithi098' 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73580b64d0 msgr2=0x7f73580b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73580b64d0 0x7f73580b89e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f737006a380 tx=0x7f736400a4c0 comp rx=0 tx=0).stop 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 msgr2=0x7f73700ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 0x7f73700ff670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f736000c970 tx=0x7f736000ce30 comp rx=0 tx=0).stop 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 shutdown_connections 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73580b64d0 0x7f73580b89e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7370108cf0 0x7f73701000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73701083a0 0x7f73700ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 --2- 172.21.15.98:0/2241547066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7370102a60 0x7f73700ff670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 >> 172.21.15.98:0/2241547066 conn(0x7f7370079730 msgr2=0x7f737006b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:13:46.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 shutdown_connections 2024-09-21T01:13:46.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:13:46.183+0000 7f73756c9640 1 -- 172.21.15.98:0/2241547066 wait complete. 2024-09-21T01:13:46.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:46 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:46.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:46 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.749 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:46 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.776 DEBUG:teuthology.orchestra.run.smithi098:osd.2> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T01:13:46.778 INFO:tasks.cephadm:Deploying osd.3 on smithi027 with /dev/vg_nvme/lv_3... 2024-09-21T01:13:46.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:13:46.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:46 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:46.835 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:46 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:46.836 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:46 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:47.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:46 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:13:47.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:46 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:47.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:46 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:47.166 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:13:47.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:47 smithi098 ceph-mon[34372]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:47.670 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:47 smithi027 ceph-mon[30375]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:48.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:47 smithi089 ceph-mon[34486]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:48.538 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:48 smithi098 ceph-mon[34372]: from='osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:13:48.538 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:48 smithi098 ceph-mon[34372]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:13:48.538 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2[39550]: 2024-09-21T01:13:48.238+0000 7f218e830640 -1 osd.2 0 log_to_monitors true 2024-09-21T01:13:48.766 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:48 smithi027 ceph-mon[30375]: from='osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:13:48.766 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:48 smithi027 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:13:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:48 smithi089 ceph-mon[34486]: from='osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:13:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:48 smithi089 ceph-mon[34486]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T01:13:49.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:49 smithi098 ceph-mon[34372]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:49.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:49 smithi098 ceph-mon[34372]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:13:49.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:49 smithi098 ceph-mon[34372]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T01:13:49.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:49.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:49 smithi098 ceph-mon[34372]: from='osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:13:49.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:49 smithi098 ceph-mon[34372]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:13:49.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:49 smithi027 ceph-mon[30375]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:49.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:49 smithi027 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:13:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:49 smithi027 ceph-mon[30375]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T01:13:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:49 smithi027 ceph-mon[30375]: from='osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:13:49.821 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:49 smithi027 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:13:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:49 smithi089 ceph-mon[34486]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:49 smithi089 ceph-mon[34486]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T01:13:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:49 smithi089 ceph-mon[34486]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T01:13:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:49 smithi089 ceph-mon[34486]: from='osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:13:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:49 smithi089 ceph-mon[34486]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:13:50.552 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2[39550]: 2024-09-21T01:13:50.252+0000 7f218a7b2640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T01:13:50.552 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2[39550]: 2024-09-21T01:13:50.257+0000 7f21825a8640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:13:50.581 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:13:50.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi098 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T01:13:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:50 smithi098 ceph-mon[34372]: from='osd.2 ' entity='osd.2' 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi098 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T01:13:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:50.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:50.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:50.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:50 smithi027 ceph-mon[30375]: from='osd.2 ' entity='osd.2' 2024-09-21T01:13:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi098 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi098 to 17088M 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:13:51.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:51.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:50 smithi089 ceph-mon[34486]: from='osd.2 ' entity='osd.2' 2024-09-21T01:13:51.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 -- 172.21.15.27:0/2307115858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810b6d0 msgr2=0x7f1ec810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:51.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2307115858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810b6d0 0x7f1ec810bae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009900 tx=0x7f1eb402a410 comp rx=0 tx=0).stop 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 -- 172.21.15.27:0/2307115858 shutdown_connections 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2307115858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec810c020 0x7f1ec8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2307115858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810b6d0 0x7f1ec810bae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2307115858 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec8105d90 0x7f1ec81061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 -- 172.21.15.27:0/2307115858 >> 172.21.15.27:0/2307115858 conn(0x7f1ec8101860 msgr2=0x7f1ec8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 -- 172.21.15.27:0/2307115858 shutdown_connections 2024-09-21T01:13:51.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.663+0000 7f1ed0400640 1 -- 172.21.15.27:0/2307115858 wait complete. 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.664+0000 7f1ed0400640 1 Processor -- start 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 -- start start 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec8105d90 0x7f1ec819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 0x7f1ec819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810c020 0x7f1ec81a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8115d30 con 0x7f1ec810c020 2024-09-21T01:13:51.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8115bb0 con 0x7f1ec810b6d0 2024-09-21T01:13:51.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ed0400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8115eb0 con 0x7f1ec8105d90 2024-09-21T01:13:51.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ecd000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 0x7f1ec819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:51.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ece400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810c020 0x7f1ec81a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:51.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.665+0000 7f1ecd000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 0x7f1ec819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58490/0 (socket says 172.21.15.27:58490) 2024-09-21T01:13:51.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 -- 172.21.15.27:0/2058458779 learned_addr learned my addr 172.21.15.27:0/2058458779 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:13:51.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecda00640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec8105d90 0x7f1ec819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 -- 172.21.15.27:0/2058458779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec8105d90 msgr2=0x7f1ec819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec8105d90 0x7f1ec819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 -- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810c020 msgr2=0x7f1ec81a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810c020 0x7f1ec81a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 -- 172.21.15.27:0/2058458779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a2730 con 0x7f1ec810b6d0 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.666+0000 7f1ecd000640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 0x7f1ec819cd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005ab0 tx=0x7f1eb4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.667+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb403f070 con 0x7f1ec810b6d0 2024-09-21T01:13:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.667+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb4031b50 con 0x7f1ec810b6d0 2024-09-21T01:13:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.667+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a29c0 con 0x7f1ec810b6d0 2024-09-21T01:13:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.667+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4035030 con 0x7f1ec810b6d0 2024-09-21T01:13:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.667+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a9180 con 0x7f1ec810b6d0 2024-09-21T01:13:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.668+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec81061a0 con 0x7f1ec810b6d0 2024-09-21T01:13:51.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.670+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 20) ==== 154473+0+0 (secure 0 0 0) 0x7f1eb4042070 con 0x7f1ec810b6d0 2024-09-21T01:13:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.673+0000 7f1ec6000640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eb00b6550 0x7f1eb00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:13:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.673+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3066+0+0 (secure 0 0 0) 0x7f1eb41067a0 con 0x7f1ec810b6d0 2024-09-21T01:13:51.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.673+0000 7f1ecda00640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eb00b6550 0x7f1eb00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:13:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.673+0000 7f1ecda00640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eb00b6550 0x7f1eb00b8a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009bd0 tx=0x7f1ebc005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:13:51.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.674+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1eb40d28d0 con 0x7f1ec810b6d0 2024-09-21T01:13:51.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:51.881+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1ec8063800 con 0x7f1eb00b6550 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] boot 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T01:13:51.882 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:51.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:13:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:13:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] boot 2024-09-21T01:13:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T01:13:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: osd.2 [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] boot 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T01:13:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T01:13:52.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:52 smithi027 ceph-mon[30375]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:52.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:52.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:52.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:52 smithi098 ceph-mon[34372]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:53.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:52 smithi089 ceph-mon[34486]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:13:53.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:13:53.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:13:53.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:13:53.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:53 smithi027 ceph-mon[30375]: pgmap v73: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:53.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:53 smithi027 ceph-mon[30375]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T01:13:53.845 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:53 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:13:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:53 smithi098 ceph-mon[34372]: pgmap v73: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:53 smithi098 ceph-mon[34372]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T01:13:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:53 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:13:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:53 smithi089 ceph-mon[34486]: pgmap v73: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:53 smithi089 ceph-mon[34486]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T01:13:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:53 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:13:54.814 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:13:54.814 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 ceph-mon[30375]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T01:13:54.814 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51398]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51398]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51398]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51402]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51402]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:54.815 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51402]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:54.839 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:54 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:13:54.840 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:54 smithi098 ceph-mon[34372]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T01:13:54.840 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:54 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:13:55.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:54 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T01:13:55.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:54 smithi089 ceph-mon[34486]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T01:13:55.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:54 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:13:55.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42606]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:13:55.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42606]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42606]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42606]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.001 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42609]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:13:55.001 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42609]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.001 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42609]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.001 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:13:54 smithi089 sudo[42609]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43466]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43466]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43466]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43466]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43469]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43469]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43469]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.148 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43469]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43472]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43472]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:54 smithi098 sudo[43472]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 sudo[43472]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 sudo[43475]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 sudo[43475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 sudo[43475]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 sudo[43475]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51410]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51410]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51410]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51410]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51413]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51413]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51413]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:54 smithi027 sudo[51413]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42612]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42612]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42612]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42612]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42615]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T01:13:55.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 sudo[42615]: pam_unix(sudo:session): session closed for user root 2024-09-21T01:13:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1144251434' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae443ec8-2955-493e-a422-395e3948b860"} : dispatch 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1144251434' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae443ec8-2955-493e-a422-395e3948b860"}]': finished 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:13:55.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:13:55.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:55.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:13:55.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1144251434' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae443ec8-2955-493e-a422-395e3948b860"} : dispatch 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1144251434' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae443ec8-2955-493e-a422-395e3948b860"}]': finished 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:13:55.935 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:13:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:13:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T01:13:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T01:13:56.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T01:13:56.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T01:13:56.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T01:13:56.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1144251434' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae443ec8-2955-493e-a422-395e3948b860"} : dispatch 2024-09-21T01:13:56.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1144251434' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae443ec8-2955-493e-a422-395e3948b860"}]': finished 2024-09-21T01:13:56.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T01:13:56.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:13:56.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:13:56.596+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f1eb40ca110 con 0x7f1ec810b6d0 2024-09-21T01:13:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:56 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1135679336' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:56.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:56 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1135679336' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:56.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:56 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1135679336' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:13:57.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:57 smithi098 ceph-mon[34372]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:57.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:57 smithi098 ceph-mon[34372]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T01:13:57.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:57 smithi027 ceph-mon[30375]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:57.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:57 smithi027 ceph-mon[30375]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T01:13:57.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:57 smithi089 ceph-mon[34486]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:57 smithi089 ceph-mon[34486]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T01:13:59.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:13:59 smithi027 ceph-mon[30375]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:59.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:13:59 smithi098 ceph-mon[34372]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:13:59.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:13:59 smithi089 ceph-mon[34486]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:01.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:01 smithi098 ceph-mon[34372]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:01 smithi027 ceph-mon[30375]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:01.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:01 smithi089 ceph-mon[34486]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:03.694 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:03 smithi027 ceph-mon[30375]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:03.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:03 smithi098 ceph-mon[34372]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:03.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:03 smithi089 ceph-mon[34486]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:05 smithi098 ceph-mon[34372]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:14:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:05 smithi098 ceph-mon[34372]: Deploying daemon osd.3 on smithi027 2024-09-21T01:14:05.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:05 smithi027 ceph-mon[30375]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:05.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:14:05.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:05.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:05 smithi027 ceph-mon[30375]: Deploying daemon osd.3 on smithi027 2024-09-21T01:14:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:05 smithi089 ceph-mon[34486]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T01:14:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:05 smithi089 ceph-mon[34486]: Deploying daemon osd.3 on smithi027 2024-09-21T01:14:07.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:07 smithi098 ceph-mon[34372]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:07.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:07 smithi027 ceph-mon[30375]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:08.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:07 smithi089 ceph-mon[34486]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:09.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:09 smithi027 ceph-mon[30375]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:09.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:09 smithi098 ceph-mon[34372]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:09 smithi089 ceph-mon[34486]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:11.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:11 smithi027 ceph-mon[30375]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:11.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:11 smithi098 ceph-mon[34372]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:11 smithi089 ceph-mon[34486]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:13.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:13 smithi098 ceph-mon[34372]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:13.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:13 smithi027 ceph-mon[30375]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:13.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:13 smithi089 ceph-mon[34486]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:14.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:14.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:14.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:14.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:14.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:14.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:15.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:15.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:15.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:15 smithi089 ceph-mon[34486]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:16.139 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:15 smithi027 ceph-mon[30375]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:16.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:15 smithi098 ceph-mon[34372]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:17 smithi098 ceph-mon[34372]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:17 smithi027 ceph-mon[30375]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:17 smithi089 ceph-mon[34486]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:19.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.177+0000 7f1ec6000640 1 -- 172.21.15.27:0/2058458779 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ec8063800 con 0x7f1eb00b6550 2024-09-21T01:14:19.178 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 3 on host 'smithi027' 2024-09-21T01:14:19.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.179+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eb00b6550 msgr2=0x7f1eb00b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:19.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.179+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eb00b6550 0x7f1eb00b8a60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009bd0 tx=0x7f1ebc005f90 comp rx=0 tx=0).stop 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.179+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 msgr2=0x7f1ec819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.179+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 0x7f1ec819cd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005ab0 tx=0x7f1eb4003b90 comp rx=0 tx=0).stop 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 shutdown_connections 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1eb00b6550 0x7f1eb00b8a60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec810c020 0x7f1ec81a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1ec810b6d0 0x7f1ec819cd60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 --2- 172.21.15.27:0/2058458779 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1ec8105d90 0x7f1ec819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 >> 172.21.15.27:0/2058458779 conn(0x7f1ec8101860 msgr2=0x7f1ec8108ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 shutdown_connections 2024-09-21T01:14:19.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:14:19.180+0000 7f1ed0400640 1 -- 172.21.15.27:0/2058458779 wait complete. 2024-09-21T01:14:19.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:19 smithi098 ceph-mon[34372]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:19.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:19.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:19.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:19 smithi027 ceph-mon[30375]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:19.994 DEBUG:teuthology.orchestra.run.smithi027:osd.3> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T01:14:19.998 INFO:tasks.cephadm:Deploying osd.4 on smithi089 with /dev/vg_nvme/lv_3... 2024-09-21T01:14:19.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:14:20.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:19 smithi089 ceph-mon[34486]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:20.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:20.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:20.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:20.252 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:14:20.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:20 smithi089 ceph-mon[34486]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:14:20.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:20 smithi098 ceph-mon[34372]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:14:20.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:20 smithi027 ceph-mon[30375]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T01:14:21.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:21 smithi027 ceph-mon[30375]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:21.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:21 smithi027 ceph-mon[30375]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:14:21.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:21 smithi027 ceph-mon[30375]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T01:14:21.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:21 smithi027 ceph-mon[30375]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:14:21.635 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:21.660 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:21 smithi089 ceph-mon[34486]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:21.660 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:21 smithi089 ceph-mon[34486]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:14:21.660 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:21 smithi089 ceph-mon[34486]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T01:14:21.660 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:21 smithi089 ceph-mon[34486]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:14:21.661 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:21 smithi098 ceph-mon[34372]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:21 smithi098 ceph-mon[34372]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T01:14:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:21 smithi098 ceph-mon[34372]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T01:14:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:21 smithi098 ceph-mon[34372]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:14:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:14:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T01:14:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' 2024-09-21T01:14:22.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] boot 2024-09-21T01:14:22.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-mon[30375]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:14:22.682 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:14:22.399+0000 7f5503400640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T01:14:22.682 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:14:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:14:22.421+0000 7f54f9400640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:14:22.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:14:22.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T01:14:22.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' 2024-09-21T01:14:22.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] boot 2024-09-21T01:14:22.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:22 smithi089 ceph-mon[34486]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:14:22.874 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: from='osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168]' entity='osd.3' 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: osd.3 [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] boot 2024-09-21T01:14:22.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:22 smithi098 ceph-mon[34372]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T01:14:22.899 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2024-09-21T01:14:23.149 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:14:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 -- 172.21.15.89:0/4220808719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30101df0 msgr2=0x7f4c30102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:23.652 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 --2- 172.21.15.89:0/4220808719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30101df0 0x7f4c30102200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009900 tx=0x7f4c2402a480 comp rx=0 tx=0).stop 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 -- 172.21.15.89:0/4220808719 shutdown_connections 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 --2- 172.21.15.89:0/4220808719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c30102940 0x7f4c301125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 --2- 172.21.15.89:0/4220808719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30101df0 0x7f4c30102200 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 --2- 172.21.15.89:0/4220808719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30104710 0x7f4c30104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.650+0000 7f4c3761d640 1 -- 172.21.15.89:0/4220808719 >> 172.21.15.89:0/4220808719 conn(0x7f4c30079ec0 msgr2=0x7f4c3007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.651+0000 7f4c3761d640 1 -- 172.21.15.89:0/4220808719 shutdown_connections 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.651+0000 7f4c3761d640 1 -- 172.21.15.89:0/4220808719 wait complete. 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.651+0000 7f4c3761d640 1 Processor -- start 2024-09-21T01:14:23.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.651+0000 7f4c3761d640 1 -- start start 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c3761d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c30101df0 0x7f4c3019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c3761d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30102940 0x7f4c3019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c3761d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 0x7f4c301a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c3761d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30115d30 con 0x7f4c30101df0 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c3761d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30115bb0 con 0x7f4c30104710 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c3761d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c30115eb0 con 0x7f4c30102940 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 0x7f4c301a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 0x7f4c301a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:35920/0 (socket says 172.21.15.89:35920) 2024-09-21T01:14:23.655 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 -- 172.21.15.89:0/1571833316 learned_addr learned my addr 172.21.15.89:0/1571833316 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c34bb8640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30102940 0x7f4c3019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 -- 172.21.15.89:0/1571833316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30102940 msgr2=0x7f4c3019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30102940 0x7f4c3019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 -- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c30101df0 msgr2=0x7f4c3019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c30101df0 0x7f4c3019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 -- 172.21.15.89:0/1571833316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c301a2690 con 0x7f4c30104710 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c34bb8640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30102940 0x7f4c3019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.652+0000 7f4c35bba640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 0x7f4c301a1f60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800eaa0 tx=0x7f4c2800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.653+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800cdb0 con 0x7f4c30104710 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.653+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28005020 con 0x7f4c30104710 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.653+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280116d0 con 0x7f4c30104710 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.653+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c301a2980 con 0x7f4c30104710 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.653+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c301a90e0 con 0x7f4c30104710 2024-09-21T01:14:23.656 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.654+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c30102200 con 0x7f4c30104710 2024-09-21T01:14:23.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.655+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4c280051d0 con 0x7f4c30104710 2024-09-21T01:14:23.657 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.656+0000 7f4c227fc640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4c1c0b65e0 0x7f4c1c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:23.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.657+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3884+0+0 (secure 0 0 0) 0x7f4c280e6890 con 0x7f4c30104710 2024-09-21T01:14:23.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.657+0000 7f4c353b9640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4c1c0b65e0 0x7f4c1c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:23.658 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.657+0000 7f4c353b9640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4c1c0b65e0 0x7f4c1c0b8af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005fd0 tx=0x7f4c2c005850 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:14:23.661 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.660+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c28016030 con 0x7f4c30104710 2024-09-21T01:14:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:23 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:14:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:23 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:14:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:23 smithi027 ceph-mon[30375]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:23 smithi027 ceph-mon[30375]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T01:14:23.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:23.847+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4c301014f0 con 0x7f4c1c0b65e0 2024-09-21T01:14:23.848 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:23 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:14:23.848 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:23 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:14:23.848 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:23 smithi089 ceph-mon[34486]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:23.848 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:23.848 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:23 smithi089 ceph-mon[34486]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T01:14:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:23 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:14:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:23 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:14:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:23 smithi098 ceph-mon[34372]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T01:14:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T01:14:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:23 smithi098 ceph-mon[34372]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T01:14:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi027 2024-09-21T01:14:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:25.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:24 smithi098 ceph-mon[34372]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T01:14:25.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi027 2024-09-21T01:14:25.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:25.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:25.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:25.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:24 smithi027 ceph-mon[30375]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T01:14:25.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi027 2024-09-21T01:14:25.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:25.214 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi027 to 8544M 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:25.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:24 smithi089 ceph-mon[34486]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T01:14:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:25 smithi098 ceph-mon[34372]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:25 smithi027 ceph-mon[30375]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:25 smithi089 ceph-mon[34486]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:26 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/1115425713' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"} : dispatch 2024-09-21T01:14:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:26 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"} : dispatch 2024-09-21T01:14:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:26 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"}]': finished 2024-09-21T01:14:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:26 smithi098 ceph-mon[34372]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:14:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:26 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/1115425713' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"} : dispatch 2024-09-21T01:14:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:26 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"} : dispatch 2024-09-21T01:14:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:26 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"}]': finished 2024-09-21T01:14:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:26 smithi027 ceph-mon[30375]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:14:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:26 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/1115425713' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"} : dispatch 2024-09-21T01:14:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:26 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"} : dispatch 2024-09-21T01:14:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:26 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00c2670f-c063-4152-8f27-c437a83f1e03"}]': finished 2024-09-21T01:14:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:26 smithi089 ceph-mon[34486]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T01:14:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:27 smithi098 ceph-mon[34372]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:27 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/2707139911' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:14:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:27 smithi027 ceph-mon[30375]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:27 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/2707139911' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:14:28.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:27 smithi089 ceph-mon[34486]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:27 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/2707139911' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:14:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:29 smithi098 ceph-mon[34372]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T01:14:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:29 smithi027 ceph-mon[30375]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T01:14:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:29 smithi089 ceph-mon[34486]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T01:14:31.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:31 smithi089 ceph-mon[34486]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T01:14:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:31 smithi098 ceph-mon[34372]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T01:14:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:31 smithi027 ceph-mon[30375]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-21T01:14:34.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:33 smithi089 ceph-mon[34486]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:14:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:33 smithi098 ceph-mon[34372]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:14:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:33 smithi027 ceph-mon[30375]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:14:35.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:34 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:14:35.090 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:34 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:35.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:34 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:14:35.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:34 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:35.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:34 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T01:14:35.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:34 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:35.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:35 smithi089 ceph-mon[34486]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:14:36.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:35 smithi089 ceph-mon[34486]: Deploying daemon osd.4 on smithi089 2024-09-21T01:14:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:35 smithi098 ceph-mon[34372]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:14:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:35 smithi098 ceph-mon[34372]: Deploying daemon osd.4 on smithi089 2024-09-21T01:14:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:35 smithi027 ceph-mon[30375]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-21T01:14:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:35 smithi027 ceph-mon[30375]: Deploying daemon osd.4 on smithi089 2024-09-21T01:14:38.063 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:37 smithi089 ceph-mon[34486]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T01:14:38.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:37 smithi098 ceph-mon[34372]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T01:14:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:37 smithi027 ceph-mon[30375]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T01:14:40.128 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:39 smithi089 ceph-mon[34486]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:14:40.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:39 smithi098 ceph-mon[34372]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:14:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:39 smithi027 ceph-mon[30375]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:14:41.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:41 smithi089 ceph-mon[34486]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:41.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:41.987 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:41.987 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:41 smithi098 ceph-mon[34372]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:41 smithi027 ceph-mon[30375]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:43 smithi098 ceph-mon[34372]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:43 smithi027 ceph-mon[30375]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:43 smithi089 ceph-mon[34486]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:45.030 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.029+0000 7f4c227fc640 1 -- 172.21.15.89:0/1571833316 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c301014f0 con 0x7f4c1c0b65e0 2024-09-21T01:14:45.031 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 4 on host 'smithi089' 2024-09-21T01:14:45.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4c1c0b65e0 msgr2=0x7f4c1c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4c1c0b65e0 0x7f4c1c0b8af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005fd0 tx=0x7f4c2c005850 comp rx=0 tx=0).stop 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 msgr2=0x7f4c301a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 0x7f4c301a1f60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800eaa0 tx=0x7f4c2800ef60 comp rx=0 tx=0).stop 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 shutdown_connections 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4c1c0b65e0 0x7f4c1c0b8af0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c30104710 0x7f4c301a1f60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c30102940 0x7f4c3019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 --2- 172.21.15.89:0/1571833316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c30101df0 0x7f4c3019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.031+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 >> 172.21.15.89:0/1571833316 conn(0x7f4c30079ec0 msgr2=0x7f4c30100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.032+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 shutdown_connections 2024-09-21T01:14:45.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:14:45.032+0000 7f4c3761d640 1 -- 172.21.15.89:0/1571833316 wait complete. 2024-09-21T01:14:45.670 DEBUG:teuthology.orchestra.run.smithi089:osd.4> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T01:14:45.672 INFO:tasks.cephadm:Deploying osd.5 on smithi098 with /dev/vg_nvme/lv_3... 2024-09-21T01:14:45.673 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T01:14:45.919 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:45 smithi089 ceph-mon[34486]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:45.919 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:45.919 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:45.919 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:14:46.042 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:45 smithi098 ceph-mon[34372]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:46.043 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:46.043 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:46.043 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:45 smithi027 ceph-mon[30375]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:14:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:47.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:46 smithi089 ceph-mon[34486]: from='osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:14:47.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:46 smithi089 ceph-mon[34486]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:14:47.000 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:14:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:14:46.719+0000 7f93a0bb4640 -1 osd.4 0 log_to_monitors true 2024-09-21T01:14:47.059 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:46 smithi098 ceph-mon[34372]: from='osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:14:47.059 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:46 smithi098 ceph-mon[34372]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:14:47.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:46 smithi027 ceph-mon[30375]: from='osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:14:47.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:46 smithi027 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T01:14:48.136 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:47 smithi098 ceph-mon[34372]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:48.136 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:47 smithi098 ceph-mon[34372]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:14:48.136 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:47 smithi098 ceph-mon[34372]: from='osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:14:48.136 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:47 smithi098 ceph-mon[34372]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T01:14:48.136 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:48.136 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:47 smithi098 ceph-mon[34372]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:14:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:47 smithi027 ceph-mon[30375]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:47 smithi027 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:14:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:47 smithi027 ceph-mon[30375]: from='osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:14:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:47 smithi027 ceph-mon[30375]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T01:14:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:47 smithi027 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:14:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:47 smithi089 ceph-mon[34486]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:47 smithi089 ceph-mon[34486]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T01:14:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:47 smithi089 ceph-mon[34486]: from='osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:14:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:47 smithi089 ceph-mon[34486]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T01:14:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:47 smithi089 ceph-mon[34486]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:14:48.692 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:14:48.597+0000 7f939cb36640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T01:14:48.692 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:14:48.605+0000 7f939492c640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:14:48.913 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:14:48.938 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='osd.4 ' entity='osd.4' 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:49.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.181 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='osd.4 ' entity='osd.4' 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:49.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:49.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:49.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:49.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.301 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:14:49.301 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='osd.4 ' entity='osd.4' 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:14:49.302 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 -- 172.21.15.98:0/67438898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa341094a0 msgr2=0x7faa341098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 --2- 172.21.15.98:0/67438898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa341094a0 0x7faa341098b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa20009900 tx=0x7faa2002a410 comp rx=0 tx=0).stop 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 -- 172.21.15.98:0/67438898 shutdown_connections 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 --2- 172.21.15.98:0/67438898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 0x7faa34110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 --2- 172.21.15.98:0/67438898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa341094a0 0x7faa341098b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 --2- 172.21.15.98:0/67438898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa34103b60 0x7faa34103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 -- 172.21.15.98:0/67438898 >> 172.21.15.98:0/67438898 conn(0x7faa3407b850 msgr2=0x7faa3407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 -- 172.21.15.98:0/67438898 shutdown_connections 2024-09-21T01:14:49.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 -- 172.21.15.98:0/67438898 wait complete. 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 Processor -- start 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.690+0000 7faa3afbd640 1 -- start start 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 0x7faa34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa341094a0 0x7faa34075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 0x7faa34076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faa34113ab0 con 0x7faa341094a0 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faa34113930 con 0x7faa34109df0 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa34113c30 con 0x7faa34103b60 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 0x7faa34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 0x7faa34075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52446/0 (socket says 172.21.15.98:52446) 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 -- 172.21.15.98:0/3212984853 learned_addr learned my addr 172.21.15.98:0/3212984853 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3955a640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 0x7faa34076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:49.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa33fff640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa341094a0 0x7faa34075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 -- 172.21.15.98:0/3212984853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 msgr2=0x7faa34076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 0x7faa34076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 -- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa341094a0 msgr2=0x7faa34075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa341094a0 0x7faa34075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 -- 172.21.15.98:0/3212984853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa341a8720 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3955a640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 0x7faa34076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa38d59640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 0x7faa34075710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa2800c890 tx=0x7faa2800cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28004220 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa280043c0 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa341a8a10 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.691+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28004a00 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.692+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa341a8f20 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.693+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa34103f70 con 0x7faa34103b60 2024-09-21T01:14:49.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.694+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7faa280079b0 con 0x7faa34103b60 2024-09-21T01:14:49.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.695+0000 7faa31ffb640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faa240b6560 0x7faa240b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:14:49.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.695+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4316+0+0 (secure 0 0 0) 0x7faa280e66a0 con 0x7faa34103b60 2024-09-21T01:14:49.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.695+0000 7faa33fff640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faa240b6560 0x7faa240b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:14:49.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.697+0000 7faa33fff640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faa240b6560 0x7faa240b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa20009230 tx=0x7faa20031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:14:49.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.699+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa28014030 con 0x7faa34103b60 2024-09-21T01:14:49.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:14:49.886+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faa34063800 con 0x7faa240b6560 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi089 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] boot 2024-09-21T01:14:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T01:14:50.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:50.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:14:50.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:14:50.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:49 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:14:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:14:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi089 2024-09-21T01:14:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] boot 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:14:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi089 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi089 to 8543M 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: osd.4 [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] boot 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T01:14:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T01:14:50.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:14:50.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:14:50.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:14:51.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:50 smithi027 ceph-mon[30375]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:14:51.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:50 smithi027 ceph-mon[30375]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T01:14:51.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:50 smithi089 ceph-mon[34486]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:14:51.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:50 smithi089 ceph-mon[34486]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T01:14:51.372 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:50 smithi098 ceph-mon[34372]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:14:51.372 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:50 smithi098 ceph-mon[34372]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T01:14:52.060 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:51 smithi098 ceph-mon[34372]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:51 smithi027 ceph-mon[30375]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:52.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:51 smithi089 ceph-mon[34486]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:53.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:52 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/2788252588' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"} : dispatch 2024-09-21T01:14:53.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:52 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"} : dispatch 2024-09-21T01:14:53.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:52 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"}]': finished 2024-09-21T01:14:53.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:52 smithi027 ceph-mon[30375]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:14:53.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:14:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:52 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/2788252588' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"} : dispatch 2024-09-21T01:14:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:52 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"} : dispatch 2024-09-21T01:14:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:52 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"}]': finished 2024-09-21T01:14:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:52 smithi089 ceph-mon[34486]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:14:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:14:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:52 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/2788252588' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"} : dispatch 2024-09-21T01:14:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:52 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"} : dispatch 2024-09-21T01:14:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:52 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8fa79ae-691d-44f3-b727-db027e158d64"}]': finished 2024-09-21T01:14:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:52 smithi098 ceph-mon[34372]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T01:14:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:14:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:53 smithi027 ceph-mon[30375]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:53 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/2940787112' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:14:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:53 smithi089 ceph-mon[34486]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:53 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/2940787112' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:14:54.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:53 smithi098 ceph-mon[34372]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:53 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/2940787112' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:14:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:55 smithi027 ceph-mon[30375]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:55 smithi089 ceph-mon[34486]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:55 smithi098 ceph-mon[34372]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:58.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:57 smithi027 ceph-mon[30375]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:58.211 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:57 smithi098 ceph-mon[34372]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:14:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:57 smithi089 ceph-mon[34486]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:14:59 smithi098 ceph-mon[34372]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:00.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:14:59 smithi089 ceph-mon[34486]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:14:59 smithi027 ceph-mon[30375]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:01.074 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:15:01.074 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:15:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T01:15:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:02.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:01 smithi098 ceph-mon[34372]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:01 smithi098 ceph-mon[34372]: Deploying daemon osd.5 on smithi098 2024-09-21T01:15:02.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:01 smithi089 ceph-mon[34486]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:01 smithi089 ceph-mon[34486]: Deploying daemon osd.5 on smithi098 2024-09-21T01:15:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:01 smithi027 ceph-mon[30375]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:01 smithi027 ceph-mon[30375]: Deploying daemon osd.5 on smithi098 2024-09-21T01:15:04.048 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:03 smithi098 ceph-mon[34372]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:03 smithi089 ceph-mon[34486]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:04.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:03 smithi027 ceph-mon[30375]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:05 smithi098 ceph-mon[34372]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:05 smithi089 ceph-mon[34486]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:05 smithi027 ceph-mon[30375]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:07.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:06 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:07.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:06 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:07.246 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:06 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:06 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:06 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:06 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:07.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:06 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:07.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:06 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:07.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:06 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:07 smithi089 ceph-mon[34486]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:08.355 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:07 smithi098 ceph-mon[34372]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:07 smithi027 ceph-mon[30375]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:09 smithi089 ceph-mon[34486]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:09 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:09 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:09 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:09 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:09 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:09 smithi098 ceph-mon[34372]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:10.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:09 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:09 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:09 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:10.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:09 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:10.335 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:09 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:09 smithi027 ceph-mon[30375]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:09 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:09 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:09 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:09 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:09 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.076+0000 7faa31ffb640 1 -- 172.21.15.98:0/3212984853 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa34063800 con 0x7faa240b6560 2024-09-21T01:15:11.077 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 5 on host 'smithi098' 2024-09-21T01:15:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faa240b6560 msgr2=0x7faa240b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faa240b6560 0x7faa240b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa20009230 tx=0x7faa20031400 comp rx=0 tx=0).stop 2024-09-21T01:15:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 msgr2=0x7faa34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 0x7faa34075710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7faa2800c890 tx=0x7faa2800cd50 comp rx=0 tx=0).stop 2024-09-21T01:15:11.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 shutdown_connections 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faa240b6560 0x7faa240b8a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa34109df0 0x7faa34076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa341094a0 0x7faa34075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 --2- 172.21.15.98:0/3212984853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa34103b60 0x7faa34075710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.078+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 >> 172.21.15.98:0/3212984853 conn(0x7faa3407b850 msgr2=0x7faa34107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.079+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 shutdown_connections 2024-09-21T01:15:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:15:11.079+0000 7faa3afbd640 1 -- 172.21.15.98:0/3212984853 wait complete. 2024-09-21T01:15:11.745 DEBUG:teuthology.orchestra.run.smithi098:osd.5> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T01:15:11.748 INFO:tasks.cephadm:Deploying osd.6 on smithi027 with /dev/vg_nvme/lv_2... 2024-09-21T01:15:11.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:15:12.140 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:15:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:11 smithi027 ceph-mon[30375]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:12.234 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:11 smithi098 ceph-mon[34372]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:12.234 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:12.234 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:12.234 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:12.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:11 smithi089 ceph-mon[34486]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:13.148 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:15:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:15:13.135+0000 7f35a4003640 -1 osd.5 0 log_to_monitors true 2024-09-21T01:15:14.059 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:13 smithi027 ceph-mon[30375]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:14.059 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:13 smithi027 ceph-mon[30375]: from='osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:15:14.059 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:13 smithi027 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:15:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:13 smithi089 ceph-mon[34486]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:13 smithi089 ceph-mon[34486]: from='osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:15:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:13 smithi089 ceph-mon[34486]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:15:14.338 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:13 smithi098 ceph-mon[34372]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:14.338 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:13 smithi098 ceph-mon[34372]: from='osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:15:14.338 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:13 smithi098 ceph-mon[34372]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:15.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:15.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:15.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:15.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:15.358 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:15.359 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T01:15:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T01:15:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:15:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:15.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:15.399 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:15:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:15:14.995+0000 7f359ff85640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T01:15:15.399 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:15:15.001+0000 7f359bd83640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:15:15.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:15:15.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-09-21T01:15:16.102 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi098 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:16.256 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:16.256 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi098 2024-09-21T01:15:16.256 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T01:15:16.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:15:16.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T01:15:16.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:16.257 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi098 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi098 to 8544M 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 -- 172.21.15.27:0/3869770663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 msgr2=0x7faaf0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 --2- 172.21.15.27:0/3869770663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf0103f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faae0009860 tx=0x7faae002a340 comp rx=0 tx=0).stop 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 -- 172.21.15.27:0/3869770663 shutdown_connections 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 --2- 172.21.15.27:0/3869770663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 0x7faaf0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 --2- 172.21.15.27:0/3869770663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 0x7faaf01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 --2- 172.21.15.27:0/3869770663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf0103f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 -- 172.21.15.27:0/3869770663 >> 172.21.15.27:0/3869770663 conn(0x7faaf00ff5d0 msgr2=0x7faaf0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:15:16.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.771+0000 7faaf5000640 1 -- 172.21.15.27:0/3869770663 shutdown_connections 2024-09-21T01:15:16.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.772+0000 7faaf5000640 1 -- 172.21.15.27:0/3869770663 wait complete. 2024-09-21T01:15:16.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 Processor -- start 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 -- start start 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 0x7faaf019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 0x7faaf01a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0113ad0 con 0x7faaf0109e10 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0113950 con 0x7faaf0103b80 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.773+0000 7faaf5000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0113c50 con 0x7faaf01094c0 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaee800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 0x7faaf01a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:41692/0 (socket says 172.21.15.27:41692) 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 -- 172.21.15.27:0/650521155 learned_addr learned my addr 172.21.15.27:0/650521155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faae5400640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 0x7faaf019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaee800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 0x7faaf01a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32842/0 (socket says 172.21.15.27:32842) 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 -- 172.21.15.27:0/650521155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 msgr2=0x7faaf019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 0x7faaf019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 -- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 msgr2=0x7faaf01a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 0x7faaf01a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faaede00640 1 -- 172.21.15.27:0/650521155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf01a2610 con 0x7faaf0103b80 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.774+0000 7faae5400640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 0x7faaf019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.775+0000 7faaede00640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf019c6a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faae0035040 tx=0x7faae0032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.775+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0037040 con 0x7faaf0103b80 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.775+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf01a28a0 con 0x7faaf0103b80 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.775+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae0041070 con 0x7faaf0103b80 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.775+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae0003850 con 0x7faaf0103b80 2024-09-21T01:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.775+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf01a9060 con 0x7faaf0103b80 2024-09-21T01:15:16.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.776+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0104e00 con 0x7faaf0103b80 2024-09-21T01:15:16.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.779+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7faae00039f0 con 0x7faaf0103b80 2024-09-21T01:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.780+0000 7faae7400640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faad40b6560 0x7faad40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.780+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4748+0+0 (secure 0 0 0) 0x7faae0106950 con 0x7faaf0103b80 2024-09-21T01:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.780+0000 7faae5400640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faad40b6560 0x7faad40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.781+0000 7faae5400640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faad40b6560 0x7faad40b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faad80096c0 tx=0x7faad8009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:15:16.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.782+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faae00cdf30 con 0x7faaf0103b80 2024-09-21T01:15:16.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:16.991+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7faaf010f8f0 con 0x7faad40b6560 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] boot 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:15:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] boot 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:15:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: osd.5 [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] boot 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:15:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:15:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:18 smithi027 ceph-mon[30375]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:18 smithi027 ceph-mon[30375]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:15:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:18 smithi027 ceph-mon[30375]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T01:15:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:18 smithi098 ceph-mon[34372]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:18 smithi098 ceph-mon[34372]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:15:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:18 smithi098 ceph-mon[34372]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T01:15:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:18 smithi089 ceph-mon[34486]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T01:15:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:18 smithi089 ceph-mon[34486]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:15:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:18 smithi089 ceph-mon[34486]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T01:15:20.021 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:20 smithi027 ceph-mon[30375]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:20.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:20 smithi098 ceph-mon[34372]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:20 smithi089 ceph-mon[34486]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:21.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/244723601' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "17a579ce-a646-47ff-abcc-9d967c03d62c"} : dispatch 2024-09-21T01:15:21.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/244723601' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17a579ce-a646-47ff-abcc-9d967c03d62c"}]': finished 2024-09-21T01:15:21.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:21 smithi027 ceph-mon[30375]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:15:21.344 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/244723601' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "17a579ce-a646-47ff-abcc-9d967c03d62c"} : dispatch 2024-09-21T01:15:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/244723601' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17a579ce-a646-47ff-abcc-9d967c03d62c"}]': finished 2024-09-21T01:15:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:21 smithi098 ceph-mon[34372]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:15:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/244723601' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "17a579ce-a646-47ff-abcc-9d967c03d62c"} : dispatch 2024-09-21T01:15:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/244723601' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17a579ce-a646-47ff-abcc-9d967c03d62c"}]': finished 2024-09-21T01:15:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:21 smithi089 ceph-mon[34486]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T01:15:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:22 smithi098 ceph-mon[34372]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:22 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2153236650' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:15:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:22 smithi027 ceph-mon[30375]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:22 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2153236650' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:15:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:22 smithi089 ceph-mon[34486]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:22 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2153236650' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:15:24.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:24 smithi098 ceph-mon[34372]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-21T01:15:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:24 smithi027 ceph-mon[30375]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-21T01:15:24.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:24 smithi089 ceph-mon[34486]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-21T01:15:26.339 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:26 smithi027 ceph-mon[30375]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:15:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:26 smithi098 ceph-mon[34372]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:15:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:26 smithi089 ceph-mon[34486]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:15:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:28 smithi098 ceph-mon[34372]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:15:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:28 smithi027 ceph-mon[30375]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:15:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:28 smithi089 ceph-mon[34486]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:15:30.278 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:30 smithi027 ceph-mon[30375]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:15:30.302 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:30 smithi089 ceph-mon[34486]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:15:30.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:30 smithi098 ceph-mon[34372]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:15:31.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:15:31.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:31.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:15:31.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:31.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T01:15:31.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:32.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:32 smithi027 ceph-mon[30375]: Deploying daemon osd.6 on smithi027 2024-09-21T01:15:32.168 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:32 smithi027 ceph-mon[30375]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:15:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:32 smithi098 ceph-mon[34372]: Deploying daemon osd.6 on smithi027 2024-09-21T01:15:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:32 smithi098 ceph-mon[34372]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:15:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:32 smithi089 ceph-mon[34486]: Deploying daemon osd.6 on smithi027 2024-09-21T01:15:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:32 smithi089 ceph-mon[34486]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:15:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:34 smithi027 ceph-mon[30375]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:15:34.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:34 smithi098 ceph-mon[34372]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:15:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:34 smithi089 ceph-mon[34486]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:15:36.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:36 smithi098 ceph-mon[34372]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:36 smithi027 ceph-mon[30375]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:36 smithi089 ceph-mon[34486]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:38 smithi027 ceph-mon[30375]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:38 smithi098 ceph-mon[34372]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:38 smithi089 ceph-mon[34486]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:40.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:40 smithi027 ceph-mon[30375]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:40.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:40.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:40.343 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:40 smithi098 ceph-mon[34372]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:40 smithi089 ceph-mon[34486]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:42.285 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:42 smithi027 ceph-mon[30375]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:42.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:42 smithi098 ceph-mon[34372]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:42.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:42 smithi089 ceph-mon[34486]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:43 smithi089 ceph-mon[34486]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:43 smithi098 ceph-mon[34372]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:43 smithi027 ceph-mon[30375]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.952+0000 7faae7400640 1 -- 172.21.15.27:0/650521155 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faaf010f8f0 con 0x7faad40b6560 2024-09-21T01:15:44.953 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 6 on host 'smithi027' 2024-09-21T01:15:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.954+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faad40b6560 msgr2=0x7faad40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.954+0000 7faaf5000640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faad40b6560 0x7faad40b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faad80096c0 tx=0x7faad8009260 comp rx=0 tx=0).stop 2024-09-21T01:15:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.954+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 msgr2=0x7faaf019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.954+0000 7faaf5000640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf019c6a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faae0035040 tx=0x7faae0032d50 comp rx=0 tx=0).stop 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 shutdown_connections 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7faad40b6560 0x7faad40b8a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf0109e10 0x7faaf01a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faaf01094c0 0x7faaf019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 --2- 172.21.15.27:0/650521155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faaf0103b80 0x7faaf019c6a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 >> 172.21.15.27:0/650521155 conn(0x7faaf00ff5d0 msgr2=0x7faaf0108b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 shutdown_connections 2024-09-21T01:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:15:44.955+0000 7faaf5000640 1 -- 172.21.15.27:0/650521155 wait complete. 2024-09-21T01:15:45.717 DEBUG:teuthology.orchestra.run.smithi027:osd.6> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T01:15:45.722 INFO:tasks.cephadm:Deploying osd.7 on smithi089 with /dev/vg_nvme/lv_2... 2024-09-21T01:15:45.722 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:15:45.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:45 smithi089 ceph-mon[34486]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:45.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:45.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:45.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:45 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:45.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:45 smithi089 ceph-mon[34486]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:15:45.770 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:45 smithi027 ceph-mon[30375]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:45.770 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:45.770 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:45.770 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:45 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:45.770 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:45 smithi027 ceph-mon[30375]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:15:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:45 smithi098 ceph-mon[34372]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:15:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:45 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:45 smithi098 ceph-mon[34372]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T01:15:45.976 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:15:46.977 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:46 smithi027 ceph-mon[30375]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:15:46.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:46 smithi027 ceph-mon[30375]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T01:15:46.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:46 smithi027 ceph-mon[30375]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:15:46.978 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:46 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:47.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:46 smithi089 ceph-mon[34486]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:15:47.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:46 smithi089 ceph-mon[34486]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T01:15:47.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:46 smithi089 ceph-mon[34486]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:15:47.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:46 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:47.303 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:15:46 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:15:46.976+0000 7fb877a00640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T01:15:47.303 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:15:46 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:15:46.988+0000 7fb872a00640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:15:47.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:46 smithi098 ceph-mon[34372]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T01:15:47.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:46 smithi098 ceph-mon[34372]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T01:15:47.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:46 smithi098 ceph-mon[34372]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:15:47.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:46 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.237 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:47 smithi089 ceph-mon[34486]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:48.238 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:47 smithi089 ceph-mon[34486]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:15:48.238 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:47 smithi089 ceph-mon[34486]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T01:15:48.238 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.238 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:47 smithi098 ceph-mon[34372]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:47 smithi098 ceph-mon[34372]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:15:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:47 smithi098 ceph-mon[34372]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T01:15:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:47 smithi027 ceph-mon[30375]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:48.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:47 smithi027 ceph-mon[30375]: from='osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:15:48.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:47 smithi027 ceph-mon[30375]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T01:15:48.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:48.541 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:15:48.569 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2024-09-21T01:15:48.858 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:15:49.101 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:48 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:15:49.101 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:48 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:15:49.101 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:48 smithi089 ceph-mon[34486]: osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] boot 2024-09-21T01:15:49.101 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:48 smithi089 ceph-mon[34486]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T01:15:49.101 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.327+0000 7f795c038640 1 -- 172.21.15.89:0/3148040093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954102a60 msgr2=0x7f7954102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.327+0000 7f795c038640 1 --2- 172.21.15.89:0/3148040093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954102a60 0x7f7954102e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7944009ca0 tx=0x7f794402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:15:49.342 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 -- 172.21.15.89:0/3148040093 shutdown_connections 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 --2- 172.21.15.89:0/3148040093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7954108cf0 0x7f795410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 --2- 172.21.15.89:0/3148040093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 0x7f79541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 --2- 172.21.15.89:0/3148040093 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954102a60 0x7f7954102e70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 -- 172.21.15.89:0/3148040093 >> 172.21.15.89:0/3148040093 conn(0x7f7954079730 msgr2=0x7f7954079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 -- 172.21.15.89:0/3148040093 shutdown_connections 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 -- 172.21.15.89:0/3148040093 wait complete. 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 Processor -- start 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.328+0000 7f795c038640 1 -- start start 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795c038640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7954102a60 0x7f795419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795c038640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 0x7f795419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795c038640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 0x7f795419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795c038640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7954113a80 con 0x7f79541083a0 2024-09-21T01:15:49.343 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795c038640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7954113900 con 0x7f7954108cf0 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795c038640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7954113c00 con 0x7f7954102a60 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 0x7f795419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 0x7f795419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:47304/0 (socket says 172.21.15.89:47304) 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 -- 172.21.15.89:0/2173374927 learned_addr learned my addr 172.21.15.89:0/2173374927 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f7959dd4640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7954102a60 0x7f795419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f79595d3640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 0x7f795419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 -- 172.21.15.89:0/2173374927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7954102a60 msgr2=0x7f795419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7954102a60 0x7f795419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 -- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 msgr2=0x7f795419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 0x7f795419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 -- 172.21.15.89:0/2173374927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79541a0440 con 0x7f7954108cf0 2024-09-21T01:15:49.344 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f79595d3640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 0x7f795419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.329+0000 7f795a5d5640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 0x7f795419fdd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f794c00d750 tx=0x7f794c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.330+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c017070 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.330+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79541a0730 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.330+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f794c0054d0 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.330+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c004c60 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.330+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79541a0c20 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.331+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7954102e70 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.332+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f794c009bd0 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.333+0000 7f7952ffd640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f793c0b6590 0x7f793c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.333+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5180+0+0 (secure 0 0 0) 0x7f794c0e64a0 con 0x7f7954108cf0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.334+0000 7f7959dd4640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f793c0b6590 0x7f793c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.334+0000 7f7959dd4640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f793c0b6590 0x7f793c0b8aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7944005e30 tx=0x7f7944009360 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:15:49.345 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.341+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79541a0c20 con 0x7f7954108cf0 2024-09-21T01:15:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:48 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:15:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:48 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:15:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:48 smithi098 ceph-mon[34372]: osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] boot 2024-09-21T01:15:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:48 smithi098 ceph-mon[34372]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T01:15:49.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:49.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:48 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:15:49.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:48 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:15:49.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:48 smithi027 ceph-mon[30375]: osd.6 [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] boot 2024-09-21T01:15:49.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:48 smithi027 ceph-mon[30375]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T01:15:49.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T01:15:49.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:15:49.532+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7954063800 con 0x7f793c0b6590 2024-09-21T01:15:50.145 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:50.178 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:50.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:49 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:49 smithi098 ceph-mon[34372]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:50.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:50.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T01:15:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T01:15:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:15:50.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:49 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:15:51.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:50 smithi089 ceph-mon[34486]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:15:51.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:50 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi027 2024-09-21T01:15:51.215 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:50 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T01:15:51.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:15:51.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi027 2024-09-21T01:15:51.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:50 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T01:15:51.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:50 smithi027 ceph-mon[30375]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:15:51.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:50 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi027 2024-09-21T01:15:51.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:50 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi027 to 5696M 2024-09-21T01:15:52.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:51 smithi089 ceph-mon[34486]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:52.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:51 smithi098 ceph-mon[34372]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:51 smithi027 ceph-mon[30375]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:52 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/1459491346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"} : dispatch 2024-09-21T01:15:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:52 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"} : dispatch 2024-09-21T01:15:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:52 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"}]': finished 2024-09-21T01:15:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:52 smithi089 ceph-mon[34486]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T01:15:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:15:53.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:52 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/2623787674' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:15:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:52 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/1459491346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"} : dispatch 2024-09-21T01:15:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:52 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"} : dispatch 2024-09-21T01:15:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:52 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"}]': finished 2024-09-21T01:15:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:52 smithi098 ceph-mon[34372]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T01:15:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:15:53.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:52 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/2623787674' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:15:53.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:52 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/1459491346' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"} : dispatch 2024-09-21T01:15:53.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:52 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"} : dispatch 2024-09-21T01:15:53.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:52 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ce20c01-8543-4ca7-8986-e2cee9ae1f14"}]': finished 2024-09-21T01:15:53.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:52 smithi027 ceph-mon[30375]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T01:15:53.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:15:53.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:52 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/2623787674' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:15:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:54 smithi098 ceph-mon[34372]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:54.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:54 smithi027 ceph-mon[30375]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:54.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:54 smithi089 ceph-mon[34486]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:56 smithi098 ceph-mon[34372]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:56.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:56 smithi027 ceph-mon[30375]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:56.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:56 smithi089 ceph-mon[34486]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:15:58 smithi089 ceph-mon[34486]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:15:58 smithi098 ceph-mon[34372]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:15:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:15:58 smithi027 ceph-mon[30375]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:00.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:00 smithi089 ceph-mon[34486]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:00 smithi098 ceph-mon[34372]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:00 smithi027 ceph-mon[30375]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:01.290 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:16:01.290 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:01.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:16:01.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T01:16:01.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:02.173 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:02 smithi089 ceph-mon[34486]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:02.174 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:02 smithi089 ceph-mon[34486]: Deploying daemon osd.7 on smithi089 2024-09-21T01:16:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:02 smithi027 ceph-mon[30375]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:02 smithi027 ceph-mon[30375]: Deploying daemon osd.7 on smithi089 2024-09-21T01:16:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:02 smithi098 ceph-mon[34372]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:02 smithi098 ceph-mon[34372]: Deploying daemon osd.7 on smithi089 2024-09-21T01:16:04.229 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:04 smithi089 ceph-mon[34486]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:04.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:04 smithi098 ceph-mon[34372]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:04.434 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:04 smithi027 ceph-mon[30375]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:06 smithi098 ceph-mon[34372]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:06 smithi027 ceph-mon[30375]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:06.475 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:06 smithi089 ceph-mon[34486]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:08 smithi098 ceph-mon[34372]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:08 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:08 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:08 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:08 smithi027 ceph-mon[30375]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:08 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:08 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:08 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:08 smithi089 ceph-mon[34486]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:08 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:08 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:08 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.306 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:10 smithi089 ceph-mon[34486]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:10.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:10.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:10.307 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:10 smithi098 ceph-mon[34372]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:10 smithi027 ceph-mon[30375]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.004 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.003+0000 7f7952ffd640 1 -- 172.21.15.89:0/2173374927 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7954063800 con 0x7f793c0b6590 2024-09-21T01:16:11.004 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 7 on host 'smithi089' 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f793c0b6590 msgr2=0x7f793c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f793c0b6590 0x7f793c0b8aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7944005e30 tx=0x7f7944009360 comp rx=0 tx=0).stop 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 msgr2=0x7f795419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 0x7f795419fdd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f794c00d750 tx=0x7f794c00dc10 comp rx=0 tx=0).stop 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 shutdown_connections 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f793c0b6590 0x7f793c0b8aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7954108cf0 0x7f795419fdd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79541083a0 0x7f795419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 --2- 172.21.15.89:0/2173374927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7954102a60 0x7f795419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:11.007 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 >> 172.21.15.89:0/2173374927 conn(0x7f7954079730 msgr2=0x7f79541014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:16:11.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 shutdown_connections 2024-09-21T01:16:11.008 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:16:11.006+0000 7f795c038640 1 -- 172.21.15.89:0/2173374927 wait complete. 2024-09-21T01:16:11.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:11.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:11 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:11.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:11 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:11 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:11.693 DEBUG:teuthology.orchestra.run.smithi089:osd.7> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T01:16:11.695 INFO:tasks.cephadm:Deploying osd.8 on smithi098 with /dev/vg_nvme/lv_2... 2024-09-21T01:16:11.695 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T01:16:11.949 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:16:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:12 smithi089 ceph-mon[34486]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:12.339 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:12 smithi098 ceph-mon[34372]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:12.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:12 smithi027 ceph-mon[30375]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:13.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:13 smithi027 ceph-mon[30375]: from='osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:16:13.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:13 smithi027 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:16:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:13 smithi089 ceph-mon[34486]: from='osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:16:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:13 smithi089 ceph-mon[34486]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:16:13.250 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:16:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:16:12.919+0000 7fe6a3bcc640 -1 osd.7 0 log_to_monitors true 2024-09-21T01:16:13.370 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:13 smithi098 ceph-mon[34372]: from='osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:16:13.370 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:13 smithi098 ceph-mon[34372]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T01:16:14.364 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:14 smithi098 ceph-mon[34372]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:14.364 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:14 smithi098 ceph-mon[34372]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T01:16:14.365 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:14 smithi098 ceph-mon[34372]: from='osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:16:14.365 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:14 smithi098 ceph-mon[34372]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T01:16:14.365 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:14.365 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:14 smithi098 ceph-mon[34372]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:16:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:14 smithi027 ceph-mon[30375]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:14 smithi027 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T01:16:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:14 smithi027 ceph-mon[30375]: from='osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:16:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:14 smithi027 ceph-mon[30375]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T01:16:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:14 smithi027 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:16:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-mon[34486]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-mon[34486]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T01:16:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-mon[34486]: from='osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:16:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-mon[34486]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T01:16:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-mon[34486]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:16:14.757 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:16:14.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2024-09-21T01:16:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:16:15.089 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:16:14.793+0000 7fe6a0363640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T01:16:15.089 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:16:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:16:14.802+0000 7fe697944640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:16:15.376 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:16:15.376 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T01:16:15.376 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='osd.7 ' entity='osd.7' 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] boot 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T01:16:15.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:16:15.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='osd.7 ' entity='osd.7' 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:15.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] boot 2024-09-21T01:16:15.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T01:16:15.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='osd.7 ' entity='osd.7' 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: osd.7 [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] boot 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T01:16:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.542+0000 7f72863c6640 1 -- 172.21.15.98:0/1400410785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7280106c20 msgr2=0x7f7280110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.542+0000 7f72863c6640 1 --2- 172.21.15.98:0/1400410785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7280106c20 0x7f7280110670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7278009ca0 tx=0x7f727802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 -- 172.21.15.98:0/1400410785 shutdown_connections 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 --2- 172.21.15.98:0/1400410785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7280106c20 0x7f7280110670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 --2- 172.21.15.98:0/1400410785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72801062d0 0x7f72801066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 --2- 172.21.15.98:0/1400410785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101ff0 0x7f7280102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 -- 172.21.15.98:0/1400410785 >> 172.21.15.98:0/1400410785 conn(0x7f7280079e30 msgr2=0x7f728007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 -- 172.21.15.98:0/1400410785 shutdown_connections 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 -- 172.21.15.98:0/1400410785 wait complete. 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 Processor -- start 2024-09-21T01:16:15.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.543+0000 7f72863c6640 1 -- start start 2024-09-21T01:16:15.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f72863c6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101ff0 0x7f7280102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:15.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f72863c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 0x7f72801034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:15.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f72863c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7280106c20 0x7f7280103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:15.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f72863c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7280113b20 con 0x7f7280106c20 2024-09-21T01:16:15.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f72863c6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f72801139a0 con 0x7f7280101ff0 2024-09-21T01:16:15.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f72863c6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7280113ca0 con 0x7f72801062d0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 0x7f72801034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 0x7f72801034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54192/0 (socket says 172.21.15.98:54192) 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 -- 172.21.15.98:0/1823795991 learned_addr learned my addr 172.21.15.98:0/1823795991 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727ffff640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101ff0 0x7f7280102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 -- 172.21.15.98:0/1823795991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101ff0 msgr2=0x7f7280102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101ff0 0x7f7280102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 -- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7280106c20 msgr2=0x7f7280103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7280106c20 0x7f7280103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 -- 172.21.15.98:0/1823795991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72801a8730 con 0x7f72801062d0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.544+0000 7f727f7fe640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 0x7f72801034d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f727400b450 tx=0x7f727400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.545+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274015020 con 0x7f72801062d0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.545+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274011d40 con 0x7f72801062d0 2024-09-21T01:16:15.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.545+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72801a8a20 con 0x7f72801062d0 2024-09-21T01:16:15.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.545+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727400f340 con 0x7f72801062d0 2024-09-21T01:16:15.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.545+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801a8f60 con 0x7f72801062d0 2024-09-21T01:16:15.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.546+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7280102400 con 0x7f72801062d0 2024-09-21T01:16:15.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.547+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f7274018020 con 0x7f72801062d0 2024-09-21T01:16:15.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.548+0000 7f727d7fa640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72680b6560 0x7f72680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:15.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.548+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5645+0+0 (secure 0 0 0) 0x7f72740e62e0 con 0x7f72801062d0 2024-09-21T01:16:15.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.548+0000 7f727ffff640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72680b6560 0x7f72680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:15.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.549+0000 7f727ffff640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72680b6560 0x7f72680b8a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72801049e0 tx=0x7f727000b410 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:16:15.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.552+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f727401f020 con 0x7f72801062d0 2024-09-21T01:16:15.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:15.738+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7280063800 con 0x7f72680b6560 2024-09-21T01:16:16.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:16:16.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:16:16.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:16.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi089 2024-09-21T01:16:16.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T01:16:16.377 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:16:16.378 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:16:16.378 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:16.378 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:16 smithi098 ceph-mon[34372]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T01:16:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:16:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:16:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:16.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi089 2024-09-21T01:16:16.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T01:16:16.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:16:16.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:16:16.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:16.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:16 smithi027 ceph-mon[30375]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi089 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi089 to 5695M 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:16 smithi089 ceph-mon[34486]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T01:16:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:17 smithi098 ceph-mon[34372]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:16:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:17 smithi098 ceph-mon[34372]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T01:16:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:17 smithi027 ceph-mon[30375]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:16:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:17 smithi027 ceph-mon[30375]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T01:16:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:17 smithi089 ceph-mon[34486]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:16:17.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:17 smithi089 ceph-mon[34486]: osdmap e50: 8 total, 8 up, 8 in 2024-09-21T01:16:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:18 smithi098 ceph-mon[34372]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:18 smithi027 ceph-mon[30375]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:18.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:18 smithi089 ceph-mon[34486]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:19 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/4075789142' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"} : dispatch 2024-09-21T01:16:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:19 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"} : dispatch 2024-09-21T01:16:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:19 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"}]': finished 2024-09-21T01:16:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:19 smithi098 ceph-mon[34372]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T01:16:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:19 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/1909977689' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:16:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:19 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/4075789142' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"} : dispatch 2024-09-21T01:16:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:19 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"} : dispatch 2024-09-21T01:16:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:19 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"}]': finished 2024-09-21T01:16:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:19 smithi027 ceph-mon[30375]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T01:16:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:19 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/1909977689' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:16:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:19 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/4075789142' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"} : dispatch 2024-09-21T01:16:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:19 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"} : dispatch 2024-09-21T01:16:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:19 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86419495-74dd-48c5-a17f-c4c4c7411658"}]': finished 2024-09-21T01:16:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:19 smithi089 ceph-mon[34486]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T01:16:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:19 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/1909977689' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:16:20.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:20 smithi098 ceph-mon[34372]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:20 smithi027 ceph-mon[30375]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:20 smithi089 ceph-mon[34486]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:22.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:22 smithi098 ceph-mon[34372]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-21T01:16:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:22 smithi027 ceph-mon[30375]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-21T01:16:22.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:22 smithi089 ceph-mon[34486]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-21T01:16:24.334 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:24 smithi098 ceph-mon[34372]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-21T01:16:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:24 smithi027 ceph-mon[30375]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-21T01:16:24.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:24 smithi089 ceph-mon[34486]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-21T01:16:26.363 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:26 smithi098 ceph-mon[34372]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:16:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:26 smithi027 ceph-mon[30375]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:16:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:26 smithi089 ceph-mon[34486]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:16:28.374 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:28 smithi098 ceph-mon[34372]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T01:16:28.374 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:28 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T01:16:28.374 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:28 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:28 smithi027 ceph-mon[30375]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T01:16:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:28 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T01:16:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:28 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:28 smithi089 ceph-mon[34486]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T01:16:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:28 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T01:16:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:28 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:29.368 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:29 smithi098 ceph-mon[34372]: Deploying daemon osd.8 on smithi098 2024-09-21T01:16:29.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:29 smithi027 ceph-mon[30375]: Deploying daemon osd.8 on smithi098 2024-09-21T01:16:29.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:29 smithi089 ceph-mon[34486]: Deploying daemon osd.8 on smithi098 2024-09-21T01:16:30.349 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:30 smithi098 ceph-mon[34372]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T01:16:30.401 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:30 smithi089 ceph-mon[34486]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T01:16:30.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:30 smithi027 ceph-mon[30375]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T01:16:32.264 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:32 smithi098 ceph-mon[34372]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:16:32.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:32 smithi027 ceph-mon[30375]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:16:32.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:32 smithi089 ceph-mon[34486]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:16:34.347 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:34 smithi098 ceph-mon[34372]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:34.347 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:34 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:34.348 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:34 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:34.348 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:34 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:34 smithi027 ceph-mon[30375]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:34 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:34 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:34 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:34 smithi089 ceph-mon[34486]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:34 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:34 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:34 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:36.362 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:36 smithi098 ceph-mon[34372]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:36 smithi027 ceph-mon[30375]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:36.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:36 smithi089 ceph-mon[34486]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:37.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.229+0000 7f727d7fa640 1 -- 172.21.15.98:0/1823795991 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7280063800 con 0x7f72680b6560 2024-09-21T01:16:37.230 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 8 on host 'smithi098' 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72680b6560 msgr2=0x7f72680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72680b6560 0x7f72680b8a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72801049e0 tx=0x7f727000b410 comp rx=0 tx=0).stop 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 msgr2=0x7f72801034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 0x7f72801034d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f727400b450 tx=0x7f727400b910 comp rx=0 tx=0).stop 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 shutdown_connections 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72680b6560 0x7f72680b8a70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7280106c20 0x7f7280103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f72801062d0 0x7f72801034d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 --2- 172.21.15.98:0/1823795991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7280101ff0 0x7f7280102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:37.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 >> 172.21.15.98:0/1823795991 conn(0x7f7280079e30 msgr2=0x7f72801074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:16:37.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 shutdown_connections 2024-09-21T01:16:37.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:16:37.231+0000 7f72863c6640 1 -- 172.21.15.98:0/1823795991 wait complete. 2024-09-21T01:16:37.961 DEBUG:teuthology.orchestra.run.smithi098:osd.8> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T01:16:37.963 INFO:tasks.cephadm:Deploying osd.9 on smithi027 with /dev/vg_nvme/lv_1... 2024-09-21T01:16:37.963 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T01:16:38.361 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:16:38.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:38 smithi098 ceph-mon[34372]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:38 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:38 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:38 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:38 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:38 smithi027 ceph-mon[30375]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:38 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:38 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:38 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:38 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:38 smithi089 ceph-mon[34486]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:38 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:38 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:38 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:38 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:39.422 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:39 smithi098 ceph-mon[34372]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:39.430 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:39 smithi027 ceph-mon[30375]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:39 smithi089 ceph-mon[34486]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:40.148 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:16:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:16:39.925+0000 7f71d1b59640 -1 osd.8 0 log_to_monitors true 2024-09-21T01:16:40.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:40 smithi027 ceph-mon[30375]: from='osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:16:40.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:40 smithi027 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:16:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:40 smithi089 ceph-mon[34486]: from='osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:16:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:40 smithi089 ceph-mon[34486]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:16:40.571 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:40 smithi098 ceph-mon[34372]: from='osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:16:40.571 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:40 smithi098 ceph-mon[34372]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T01:16:41.390 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:16:41.173+0000 7f71cdadb640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T01:16:41.390 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:16:41.186+0000 7f71c98d9640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:16:41.390 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-mon[34372]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T01:16:41.390 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-mon[34372]: from='osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:16:41.390 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-mon[34372]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T01:16:41.390 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:41.391 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-mon[34372]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:16:41.391 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:41 smithi098 ceph-mon[34372]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:41 smithi027 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T01:16:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:41 smithi027 ceph-mon[30375]: from='osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:16:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:41 smithi027 ceph-mon[30375]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T01:16:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:41 smithi027 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:16:41.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:41 smithi027 ceph-mon[30375]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:41.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:41 smithi089 ceph-mon[34486]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T01:16:41.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:41 smithi089 ceph-mon[34486]: from='osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:16:41.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:41 smithi089 ceph-mon[34486]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T01:16:41.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:41.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:41 smithi089 ceph-mon[34486]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:16:41.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:41 smithi089 ceph-mon[34486]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T01:16:41.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:16:41.720 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-09-21T01:16:42.118 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:16:42.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:16:42.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T01:16:42.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] boot 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi098 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:42.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:42.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:42.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:42.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:16:42.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T01:16:42.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] boot 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi098 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:42.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.503 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:42.503 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:42.503 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:42.503 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:16:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T01:16:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: osd.8 [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] boot 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi098 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi098 to 5696M 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:42.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:16:42.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:42.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:16:42.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:16:42.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 -- 172.21.15.27:0/2124827447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 msgr2=0x7f87e4112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:42.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 --2- 172.21.15.27:0/2124827447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e4112500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009ca0 tx=0x7f87dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:16:42.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 -- 172.21.15.27:0/2124827447 shutdown_connections 2024-09-21T01:16:42.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 --2- 172.21.15.27:0/2124827447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e4112500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:42.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 --2- 172.21.15.27:0/2124827447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e4076100 0x7f87e410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:42.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 --2- 172.21.15.27:0/2124827447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e40757b0 0x7f87e4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:42.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 -- 172.21.15.27:0/2124827447 >> 172.21.15.27:0/2124827447 conn(0x7f87e407b850 msgr2=0x7f87e407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:16:42.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 -- 172.21.15.27:0/2124827447 shutdown_connections 2024-09-21T01:16:42.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.746+0000 7f87e9a00640 1 -- 172.21.15.27:0/2124827447 wait complete. 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.747+0000 7f87e9a00640 1 Processor -- start 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.747+0000 7f87e9a00640 1 -- start start 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e40757b0 0x7f87e419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e4076100 0x7f87e419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e41a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e9a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4115c70 con 0x7f87e41100c0 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4115af0 con 0x7f87e40757b0 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e9a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4115df0 con 0x7f87e4076100 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e41a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e41a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35504/0 (socket says 172.21.15.27:35504) 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e3400640 1 -- 172.21.15.27:0/3076203599 learned_addr learned my addr 172.21.15.27:0/3076203599 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87da000640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e4076100 0x7f87e419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.748+0000 7f87e2a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e40757b0 0x7f87e419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e3400640 1 -- 172.21.15.27:0/3076203599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e4076100 msgr2=0x7f87e419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e3400640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e4076100 0x7f87e419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e3400640 1 -- 172.21.15.27:0/3076203599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e40757b0 msgr2=0x7f87e419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e3400640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e40757b0 0x7f87e419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e3400640 1 -- 172.21.15.27:0/3076203599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e41a25f0 con 0x7f87e41100c0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87da000640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e4076100 0x7f87e419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e2a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e40757b0 0x7f87e419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e3400640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e41a1ec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87dc005cc0 tx=0x7f87dc031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc003b70 con 0x7f87e41100c0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e41a2880 con 0x7f87e41100c0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87dc0318f0 con 0x7f87e41100c0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.749+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc02f950 con 0x7f87e41100c0 2024-09-21T01:16:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.750+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41a9040 con 0x7f87e41100c0 2024-09-21T01:16:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.751+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e4075bc0 con 0x7f87e41100c0 2024-09-21T01:16:42.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.753+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f87dc004b40 con 0x7f87e41100c0 2024-09-21T01:16:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.756+0000 7f87dbe00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87c80b6560 0x7f87c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:16:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.756+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6044+0+0 (secure 0 0 0) 0x7f87dc107e70 con 0x7f87e41100c0 2024-09-21T01:16:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.756+0000 7f87e2a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87c80b6560 0x7f87c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:16:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.757+0000 7f87e2a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87c80b6560 0x7f87c80b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009fc0 tx=0x7f87d4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:16:42.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.758+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87dc0cefd0 con 0x7f87e41100c0 2024-09-21T01:16:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:16:42.971+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f87e4101d50 con 0x7f87c80b6560 2024-09-21T01:16:43.549 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:16:43.549 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:16:43.550 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T01:16:43.550 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:43.550 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:16:43.550 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:16:43.550 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:16:43.550 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:16:43.568 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:16:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:44.572 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:44 smithi027 ceph-mon[30375]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T01:16:44.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:44 smithi098 ceph-mon[34372]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T01:16:44.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:44 smithi089 ceph-mon[34486]: osdmap e56: 9 total, 9 up, 9 in 2024-09-21T01:16:45.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:45 smithi098 ceph-mon[34372]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:45.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:45 smithi027 ceph-mon[30375]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:45.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:45 smithi089 ceph-mon[34486]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:46 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/734721317' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"} : dispatch 2024-09-21T01:16:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:46 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"} : dispatch 2024-09-21T01:16:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:46 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"}]': finished 2024-09-21T01:16:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:46 smithi027 ceph-mon[30375]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T01:16:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:46 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:16:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:46 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/734721317' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"} : dispatch 2024-09-21T01:16:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:46 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"} : dispatch 2024-09-21T01:16:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:46 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"}]': finished 2024-09-21T01:16:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:46 smithi089 ceph-mon[34486]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T01:16:46.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:46 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:16:46.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:46 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/734721317' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"} : dispatch 2024-09-21T01:16:46.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:46 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"} : dispatch 2024-09-21T01:16:46.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:46 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3a6679-2483-45b0-99b5-55c8668cb2b6"}]': finished 2024-09-21T01:16:46.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:46 smithi098 ceph-mon[34372]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T01:16:46.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:46 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:16:47.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:47 smithi027 ceph-mon[30375]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-21T01:16:47.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:47 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3334418789' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:16:47.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:47 smithi089 ceph-mon[34486]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-21T01:16:47.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:47 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3334418789' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:16:47.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:47 smithi098 ceph-mon[34372]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-21T01:16:47.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:47 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3334418789' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:16:49.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:49 smithi027 ceph-mon[30375]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-21T01:16:49.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:49 smithi089 ceph-mon[34486]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-21T01:16:49.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:49 smithi098 ceph-mon[34372]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-21T01:16:51.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:51 smithi027 ceph-mon[30375]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:16:51.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:51 smithi089 ceph-mon[34486]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:16:51.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:51 smithi098 ceph-mon[34372]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T01:16:53.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:53 smithi027 ceph-mon[30375]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:16:53.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:53 smithi089 ceph-mon[34486]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:16:53.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:53 smithi098 ceph-mon[34372]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-21T01:16:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:55 smithi027 ceph-mon[30375]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:16:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:55 smithi089 ceph-mon[34486]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:16:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:55 smithi098 ceph-mon[34372]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:16:56.619 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T01:16:56.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:56.620 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:56 smithi027 ceph-mon[30375]: Deploying daemon osd.9 on smithi027 2024-09-21T01:16:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T01:16:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:56.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:56 smithi089 ceph-mon[34486]: Deploying daemon osd.9 on smithi027 2024-09-21T01:16:56.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T01:16:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:16:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:56 smithi098 ceph-mon[34372]: Deploying daemon osd.9 on smithi027 2024-09-21T01:16:57.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:57 smithi027 ceph-mon[30375]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:57.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:57 smithi089 ceph-mon[34486]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:57.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:57 smithi098 ceph-mon[34372]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:59.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:16:59 smithi089 ceph-mon[34486]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:59.753 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:16:59 smithi027 ceph-mon[30375]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:16:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:16:59 smithi098 ceph-mon[34372]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:01.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:01 smithi089 ceph-mon[34486]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:01.850 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:01 smithi027 ceph-mon[30375]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:01.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:01 smithi098 ceph-mon[34372]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:03.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:03 smithi089 ceph-mon[34486]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:03.799 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:03 smithi027 ceph-mon[30375]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:03.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:03 smithi098 ceph-mon[34372]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:04 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:04 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:04 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:04.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:04 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:04.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:04 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:04.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:04 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:04.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:04 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:04.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:04 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:04.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:04 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:05 smithi089 ceph-mon[34486]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:05.839 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:05 smithi027 ceph-mon[30375]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:05 smithi098 ceph-mon[34372]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:07.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:07 smithi027 ceph-mon[30375]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:07.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:07.538 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:07 smithi089 ceph-mon[34486]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:07.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:07 smithi098 ceph-mon[34372]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:07.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:07.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:08.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:08 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:08.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:08 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:08.725 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:08 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:08.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:08 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:08.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:08 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:08.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:08 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:08.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:08 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:08.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:08 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:08.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:08 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:09.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.643+0000 7f87dbe00640 1 -- 172.21.15.27:0/3076203599 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87e4101d50 con 0x7f87c80b6560 2024-09-21T01:17:09.644 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 9 on host 'smithi027' 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87c80b6560 msgr2=0x7f87c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87c80b6560 0x7f87c80b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009fc0 tx=0x7f87d4005f90 comp rx=0 tx=0).stop 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 msgr2=0x7f87e41a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e41a1ec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87dc005cc0 tx=0x7f87dc031700 comp rx=0 tx=0).stop 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 shutdown_connections 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87c80b6560 0x7f87c80b8a70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e41100c0 0x7f87e41a1ec0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87e4076100 0x7f87e419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:09.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 --2- 172.21.15.27:0/3076203599 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87e40757b0 0x7f87e419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:09.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 >> 172.21.15.27:0/3076203599 conn(0x7f87e407b850 msgr2=0x7f87e4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:17:09.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 shutdown_connections 2024-09-21T01:17:09.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:17:09.646+0000 7f87e9a00640 1 -- 172.21.15.27:0/3076203599 wait complete. 2024-09-21T01:17:10.302 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:10 smithi027 ceph-mon[30375]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:10.302 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:10.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:10.303 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:10 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:10 smithi098 ceph-mon[34372]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:10 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:10.435 DEBUG:teuthology.orchestra.run.smithi027:osd.9> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T01:17:10.439 INFO:tasks.cephadm:Deploying osd.10 on smithi089 with /dev/vg_nvme/lv_1... 2024-09-21T01:17:10.439 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T01:17:10.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:10 smithi089 ceph-mon[34486]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:10.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:10.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:10.467 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:10 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:10.693 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:17:11.143 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:11 smithi027 ceph-mon[30375]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T01:17:11.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:11 smithi098 ceph-mon[34372]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T01:17:11.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:11 smithi089 ceph-mon[34486]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T01:17:12.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-mon[30375]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:12.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-mon[30375]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T01:17:12.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-mon[30375]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T01:17:12.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-mon[30375]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:17:12.213 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:12.214 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:17:12.163+0000 7f0854a00640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T01:17:12.214 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:17:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:17:12.182+0000 7f084fa00640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:17:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:12 smithi098 ceph-mon[34372]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:12 smithi098 ceph-mon[34372]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T01:17:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:12 smithi098 ceph-mon[34372]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T01:17:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:12 smithi098 ceph-mon[34372]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:17:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:12 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:12.445 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:12 smithi089 ceph-mon[34486]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:12.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:12 smithi089 ceph-mon[34486]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T01:17:12.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:12 smithi089 ceph-mon[34486]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T01:17:12.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:12 smithi089 ceph-mon[34486]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]} : dispatch 2024-09-21T01:17:12.446 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:12 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:13 smithi027 ceph-mon[30375]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:17:13.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:13 smithi027 ceph-mon[30375]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T01:17:13.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:13 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:13 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:13 smithi089 ceph-mon[34486]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:17:13.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:13 smithi089 ceph-mon[34486]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T01:17:13.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:13 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:13 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.529 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-09-21T01:17:13.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2024-09-21T01:17:13.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:13 smithi098 ceph-mon[34372]: from='osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-09-21T01:17:13.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:13 smithi098 ceph-mon[34372]: osdmap e59: 10 total, 9 up, 10 in 2024-09-21T01:17:13.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:13 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:13 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:13.848 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 -- 172.21.15.89:0/3044291165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c102a60 msgr2=0x7fab7c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 --2- 172.21.15.89:0/3044291165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c102a60 0x7fab7c102e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab78009ca0 tx=0x7fab7802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 -- 172.21.15.89:0/3044291165 shutdown_connections 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 --2- 172.21.15.89:0/3044291165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7c108cf0 0x7fab7c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 --2- 172.21.15.89:0/3044291165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab7c1083a0 0x7fab7c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 --2- 172.21.15.89:0/3044291165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c102a60 0x7fab7c102e70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 -- 172.21.15.89:0/3044291165 >> 172.21.15.89:0/3044291165 conn(0x7fab7c079730 msgr2=0x7fab7c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 -- 172.21.15.89:0/3044291165 shutdown_connections 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.368+0000 7fab83cf7640 1 -- 172.21.15.89:0/3044291165 wait complete. 2024-09-21T01:17:14.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 Processor -- start 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 -- start start 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7c102a60 0x7fab7c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab7c1083a0 0x7fab7c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 0x7fab7c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c113ad0 con 0x7fab7c102a60 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c113950 con 0x7fab7c108cf0 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab83cf7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c113c50 con 0x7fab7c1083a0 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 0x7fab7c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 0x7fab7c19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39584/0 (socket says 172.21.15.89:39584) 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 -- 172.21.15.89:0/635828701 learned_addr learned my addr 172.21.15.89:0/635828701 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab81292640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab7c1083a0 0x7fab7c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 -- 172.21.15.89:0/635828701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab7c1083a0 msgr2=0x7fab7c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:14.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab7c1083a0 0x7fab7c19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 -- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7c102a60 msgr2=0x7fab7c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7c102a60 0x7fab7c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 -- 172.21.15.89:0/635828701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c1a03d0 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.369+0000 7fab82294640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 0x7fab7c19fd60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fab74009fd0 tx=0x7fab7400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.370+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74011830 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.370+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c1a06c0 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.370+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74016070 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.370+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74010de0 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.370+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a6ee0 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.371+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c102e70 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.372+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fab740119d0 con 0x7fab7c108cf0 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.373+0000 7fab72ffd640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab680b6590 0x7fab680b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:14.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.374+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6476+0+0 (secure 0 0 0) 0x7fab740e6a00 con 0x7fab7c108cf0 2024-09-21T01:17:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.374+0000 7fab81a93640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab680b6590 0x7fab680b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.374+0000 7fab81a93640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab680b6590 0x7fab680b8aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab780094e0 tx=0x7fab78000f40 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:17:14.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.377+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab740ad920 con 0x7fab7c108cf0 2024-09-21T01:17:14.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:14 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:17:14.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:14 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:17:14.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:14 smithi089 ceph-mon[34486]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:14.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:14 smithi089 ceph-mon[34486]: osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] boot 2024-09-21T01:17:14.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:14 smithi089 ceph-mon[34486]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T01:17:14.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:14 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:17:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:14 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:17:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:14 smithi027 ceph-mon[30375]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:14 smithi027 ceph-mon[30375]: osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] boot 2024-09-21T01:17:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:14 smithi027 ceph-mon[30375]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T01:17:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:14.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:14.571+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fab7c063800 con 0x7fab680b6590 2024-09-21T01:17:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:14 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:17:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:14 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:17:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:14 smithi098 ceph-mon[34372]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-21T01:17:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:14 smithi098 ceph-mon[34372]: osd.9 [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] boot 2024-09-21T01:17:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:14 smithi098 ceph-mon[34372]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T01:17:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='client.24590 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi027 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:15.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='client.24590 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi027 2024-09-21T01:17:15.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:15.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: osdmap e61: 10 total, 10 up, 10 in 2024-09-21T01:17:15.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:15.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='client.24590 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:17:15.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:17:15.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:17:15.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi027 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi027 to 4272M 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:15.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:17.699 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:17 smithi089 ceph-mon[34486]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:17.700 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:17 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/1889686222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"} : dispatch 2024-09-21T01:17:17.700 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:17 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"} : dispatch 2024-09-21T01:17:17.700 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:17 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"}]': finished 2024-09-21T01:17:17.700 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:17 smithi089 ceph-mon[34486]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T01:17:17.700 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:17 smithi098 ceph-mon[34372]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:17 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/1889686222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"} : dispatch 2024-09-21T01:17:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:17 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"} : dispatch 2024-09-21T01:17:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:17 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"}]': finished 2024-09-21T01:17:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:17 smithi098 ceph-mon[34372]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T01:17:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:17 smithi027 ceph-mon[30375]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:17 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/1889686222' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"} : dispatch 2024-09-21T01:17:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:17 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"} : dispatch 2024-09-21T01:17:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:17 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1978e43b-cf1a-4a0c-a1bf-11a8ce290970"}]': finished 2024-09-21T01:17:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:17 smithi027 ceph-mon[30375]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T01:17:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:18.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:18 smithi089 ceph-mon[34486]: from='client.? 172.21.15.89:0/2129091800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:17:18.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:18 smithi098 ceph-mon[34372]: from='client.? 172.21.15.89:0/2129091800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:17:18.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:18 smithi027 ceph-mon[30375]: from='client.? 172.21.15.89:0/2129091800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:17:19.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:19 smithi089 ceph-mon[34486]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:19.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:19 smithi098 ceph-mon[34372]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:19 smithi027 ceph-mon[30375]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:21.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:21 smithi089 ceph-mon[34486]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:21.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:21 smithi098 ceph-mon[34372]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:21.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:21 smithi027 ceph-mon[30375]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:23.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:23 smithi089 ceph-mon[34486]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:23 smithi098 ceph-mon[34372]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:23.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:23 smithi027 ceph-mon[30375]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:25.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:25 smithi089 ceph-mon[34486]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:25.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:25 smithi098 ceph-mon[34372]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:25.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:25 smithi027 ceph-mon[30375]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:26.559 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T01:17:26.560 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:26.560 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:26 smithi089 ceph-mon[34486]: Deploying daemon osd.10 on smithi089 2024-09-21T01:17:26.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T01:17:26.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:26.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:26 smithi098 ceph-mon[34372]: Deploying daemon osd.10 on smithi089 2024-09-21T01:17:26.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T01:17:26.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:26.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:26 smithi027 ceph-mon[30375]: Deploying daemon osd.10 on smithi089 2024-09-21T01:17:27.725 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:27 smithi089 ceph-mon[34486]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:27.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:27 smithi098 ceph-mon[34372]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:27.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:27 smithi027 ceph-mon[30375]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:29.619 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:29 smithi089 ceph-mon[34486]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:29.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:29 smithi098 ceph-mon[34372]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:29.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:29 smithi027 ceph-mon[30375]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:31.608 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:31 smithi089 ceph-mon[34486]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:31.733 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:31 smithi027 ceph-mon[30375]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:31.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:31 smithi098 ceph-mon[34372]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:32.662 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:32 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:32.662 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:32 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:32.662 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:32 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:32.761 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:32 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:32.761 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:32 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:32.761 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:32 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:32.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:32 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:32.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:32 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:32.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:32 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:33 smithi027 ceph-mon[30375]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:34.247 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:33 smithi089 ceph-mon[34486]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:34.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:33 smithi098 ceph-mon[34372]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:35.635 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.633+0000 7fab72ffd640 1 -- 172.21.15.89:0/635828701 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fab7c063800 con 0x7fab680b6590 2024-09-21T01:17:35.635 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 10 on host 'smithi089' 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab680b6590 msgr2=0x7fab680b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab680b6590 0x7fab680b8aa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab780094e0 tx=0x7fab78000f40 comp rx=0 tx=0).stop 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 msgr2=0x7fab7c19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 0x7fab7c19fd60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fab74009fd0 tx=0x7fab7400b8e0 comp rx=0 tx=0).stop 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 shutdown_connections 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab680b6590 0x7fab680b8aa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab7c108cf0 0x7fab7c19fd60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:35.637 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab7c1083a0 0x7fab7c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:35.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 --2- 172.21.15.89:0/635828701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab7c102a60 0x7fab7c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:35.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 >> 172.21.15.89:0/635828701 conn(0x7fab7c079730 msgr2=0x7fab7c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:17:35.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 shutdown_connections 2024-09-21T01:17:35.638 INFO:teuthology.orchestra.run.smithi089.stderr:2024-09-21T01:17:35.636+0000 7fab83cf7640 1 -- 172.21.15.89:0/635828701 wait complete. 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:35.993 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:36.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:36.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:36.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:36.373 DEBUG:teuthology.orchestra.run.smithi089:osd.10> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T01:17:36.375 INFO:tasks.cephadm:Deploying osd.11 on smithi098 with /dev/vg_nvme/lv_1... 2024-09-21T01:17:36.375 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T01:17:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:17:37.594 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:37 smithi089 ceph-mon[34486]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:37.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:37 smithi098 ceph-mon[34372]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:37.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:37 smithi027 ceph-mon[30375]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:38.358 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:38 smithi098 ceph-mon[34372]: from='osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:17:38.359 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:38 smithi098 ceph-mon[34372]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:17:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:38 smithi089 ceph-mon[34486]: from='osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:17:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:38 smithi089 ceph-mon[34486]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:17:38.500 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:17:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:17:38.231+0000 7fc259cdc640 -1 osd.10 0 log_to_monitors true 2024-09-21T01:17:38.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:38 smithi027 ceph-mon[30375]: from='osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:17:38.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:38 smithi027 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T01:17:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:39 smithi098 ceph-mon[34372]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T01:17:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:39 smithi098 ceph-mon[34372]: from='osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:17:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:39 smithi098 ceph-mon[34372]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T01:17:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:39 smithi098 ceph-mon[34372]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:17:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:39 smithi098 ceph-mon[34372]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:39.418 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-21T01:17:39.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2024-09-21T01:17:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:39 smithi027 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T01:17:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:39 smithi027 ceph-mon[30375]: from='osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:17:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:39 smithi027 ceph-mon[30375]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T01:17:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:39 smithi027 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:17:39.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:39 smithi027 ceph-mon[30375]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:39.695 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:17:39.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-mon[34486]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T01:17:39.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-mon[34486]: from='osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:17:39.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-mon[34486]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T01:17:39.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:39.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-mon[34486]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]} : dispatch 2024-09-21T01:17:39.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-mon[34486]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-21T01:17:39.750 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:17:39.331+0000 7fc255c5e640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T01:17:39.750 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:17:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:17:39.345+0000 7fc251a5c640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 -- 172.21.15.98:0/1226058216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8108cf0 msgr2=0x7fdec810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 --2- 172.21.15.98:0/1226058216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8108cf0 0x7fdec810f520 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009ca0 tx=0x7fdec002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 -- 172.21.15.98:0/1226058216 shutdown_connections 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 --2- 172.21.15.98:0/1226058216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8108cf0 0x7fdec810f520 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 --2- 172.21.15.98:0/1226058216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec81083a0 0x7fdec81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 --2- 172.21.15.98:0/1226058216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec8102a60 0x7fdec8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 -- 172.21.15.98:0/1226058216 >> 172.21.15.98:0/1226058216 conn(0x7fdec8079730 msgr2=0x7fdec8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 -- 172.21.15.98:0/1226058216 shutdown_connections 2024-09-21T01:17:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.221+0000 7fded062e640 1 -- 172.21.15.98:0/1226058216 wait complete. 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 Processor -- start 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 -- start start 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 0x7fdec819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec81083a0 0x7fdec819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec8108cf0 0x7fdec819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81129e0 con 0x7fdec81083a0 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8112860 con 0x7fdec8108cf0 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fded062e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8112b60 con 0x7fdec8102a60 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 0x7fdec819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 0x7fdec819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52424/0 (socket says 172.21.15.98:52424) 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 -- 172.21.15.98:0/1711363468 learned_addr learned my addr 172.21.15.98:0/1711363468 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdecebcb640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec8108cf0 0x7fdec819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 -- 172.21.15.98:0/1711363468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec8108cf0 msgr2=0x7fdec819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:40.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec8108cf0 0x7fdec819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 -- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec81083a0 msgr2=0x7fdec819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdecdbc9640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec81083a0 0x7fdec819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec81083a0 0x7fdec819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdece3ca640 1 -- 172.21.15.98:0/1711363468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec81a6640 con 0x7fdec8102a60 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdecebcb640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec8108cf0 0x7fdec819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.222+0000 7fdecdbc9640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec81083a0 0x7fdec819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.223+0000 7fdece3ca640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 0x7fdec819b3b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b4a0 tx=0x7fdec400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.223+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4014070 con 0x7fdec8102a60 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.223+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec4010ac0 con 0x7fdec8102a60 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.223+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec81a6930 con 0x7fdec8102a60 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.223+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4017c30 con 0x7fdec8102a60 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.223+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a6ec0 con 0x7fdec8102a60 2024-09-21T01:17:40.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.224+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec8102e70 con 0x7fdec8102a60 2024-09-21T01:17:40.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.226+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fdec4017070 con 0x7fdec8102a60 2024-09-21T01:17:40.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.227+0000 7fdebb7fe640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdeb40b6610 0x7fdeb40b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:17:40.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.227+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6629+0+0 (secure 0 0 0) 0x7fdec40e6350 con 0x7fdec8102a60 2024-09-21T01:17:40.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.227+0000 7fdecdbc9640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdeb40b6610 0x7fdeb40b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:17:40.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.228+0000 7fdecdbc9640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdeb40b6610 0x7fdeb40b8b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdec8107ad0 tx=0x7fdebc005d30 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:17:40.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.230+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdec40ad270 con 0x7fdec8102a60 2024-09-21T01:17:40.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:17:40.425+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdec8063800 con 0x7fdeb40b6610 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi089 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T01:17:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:40.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi089 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T01:17:40.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:40.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:40.683 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-09-21T01:17:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: osdmap e64: 11 total, 10 up, 11 in 2024-09-21T01:17:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi089 2024-09-21T01:17:40.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi089 to 4271M 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:17:40.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] boot 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:41 smithi098 ceph-mon[34372]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] boot 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:17:41.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:17:41.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:17:41.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:41.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:41 smithi027 ceph-mon[30375]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: osd.10 [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] boot 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:17:41.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T01:17:41.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T01:17:41.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:41.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:41 smithi089 ceph-mon[34486]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T01:17:43.411 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T01:17:43.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:43.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/306373044' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"} : dispatch 2024-09-21T01:17:43.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"} : dispatch 2024-09-21T01:17:43.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"}]': finished 2024-09-21T01:17:43.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T01:17:43.412 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/306373044' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"} : dispatch 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"} : dispatch 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"}]': finished 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T01:17:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:17:43.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: osdmap e67: 11 total, 11 up, 11 in 2024-09-21T01:17:43.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:43.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/306373044' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"} : dispatch 2024-09-21T01:17:43.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"} : dispatch 2024-09-21T01:17:43.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0eb4e911-7e6f-4cf0-b60c-983aed5f5b06"}]': finished 2024-09-21T01:17:43.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T01:17:43.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:17:44.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:44 smithi098 ceph-mon[34372]: from='client.? 172.21.15.98:0/3824289191' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:17:44.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:44 smithi027 ceph-mon[30375]: from='client.? 172.21.15.98:0/3824289191' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:17:44.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:44 smithi089 ceph-mon[34486]: from='client.? 172.21.15.98:0/3824289191' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T01:17:45.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:45 smithi098 ceph-mon[34372]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:45.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:45 smithi027 ceph-mon[30375]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:45.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:45 smithi089 ceph-mon[34486]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:17:47.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:47 smithi089 ceph-mon[34486]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:17:47.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:47 smithi098 ceph-mon[34372]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:17:47.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:47 smithi027 ceph-mon[30375]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T01:17:49.624 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:49 smithi098 ceph-mon[34372]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T01:17:49.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:49 smithi089 ceph-mon[34486]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T01:17:49.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:49 smithi027 ceph-mon[30375]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T01:17:51.696 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:51 smithi098 ceph-mon[34372]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T01:17:51.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:51 smithi089 ceph-mon[34486]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T01:17:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:51 smithi027 ceph-mon[30375]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T01:17:52.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T01:17:52.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:52.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:52 smithi098 ceph-mon[34372]: Deploying daemon osd.11 on smithi098 2024-09-21T01:17:52.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T01:17:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:52.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:52 smithi089 ceph-mon[34486]: Deploying daemon osd.11 on smithi098 2024-09-21T01:17:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T01:17:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:17:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:52 smithi027 ceph-mon[30375]: Deploying daemon osd.11 on smithi098 2024-09-21T01:17:53.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:53 smithi098 ceph-mon[34372]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:17:53.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:53 smithi089 ceph-mon[34486]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:17:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:53 smithi027 ceph-mon[30375]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T01:17:55.586 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:55 smithi098 ceph-mon[34372]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T01:17:55.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:55 smithi089 ceph-mon[34486]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T01:17:55.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:55 smithi027 ceph-mon[30375]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T01:17:57.526 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:57 smithi098 ceph-mon[34372]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:17:57.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:57 smithi089 ceph-mon[34486]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:17:57.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:57 smithi027 ceph-mon[30375]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T01:17:58.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:58.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:58.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:58.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:58.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:58.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:58.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:17:58.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:17:58.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:00.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:17:59 smithi089 ceph-mon[34486]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:00.395 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:17:59 smithi098 ceph-mon[34372]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:17:59 smithi027 ceph-mon[30375]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:01.755 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 11 on host 'smithi098' 2024-09-21T01:18:01.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.755+0000 7fdebb7fe640 1 -- 172.21.15.98:0/1711363468 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdec8063800 con 0x7fdeb40b6610 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdeb40b6610 msgr2=0x7fdeb40b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdeb40b6610 0x7fdeb40b8b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdec8107ad0 tx=0x7fdebc005d30 comp rx=0 tx=0).stop 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 msgr2=0x7fdec819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 0x7fdec819b3b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b4a0 tx=0x7fdec400b960 comp rx=0 tx=0).stop 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 shutdown_connections 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdeb40b6610 0x7fdeb40b8b20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdec8108cf0 0x7fdec819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdec81083a0 0x7fdec819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:01.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 --2- 172.21.15.98:0/1711363468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdec8102a60 0x7fdec819b3b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:01.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 >> 172.21.15.98:0/1711363468 conn(0x7fdec8079730 msgr2=0x7fdec8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:01.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 shutdown_connections 2024-09-21T01:18:01.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:01.757+0000 7fded062e640 1 -- 172.21.15.98:0/1711363468 wait complete. 2024-09-21T01:18:02.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:02.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.123 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:02.124 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:02.124 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.124 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:02.124 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.124 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:02.481 DEBUG:teuthology.orchestra.run.smithi098:osd.11> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T01:18:02.483 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T01:18:02.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:18:02.878 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 -- 172.21.15.27:0/1676601579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10b6c0 msgr2=0x7f6dbc10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 --2- 172.21.15.27:0/1676601579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc10bad0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009900 tx=0x7f6da802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 -- 172.21.15.27:0/1676601579 shutdown_connections 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 --2- 172.21.15.27:0/1676601579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc10c010 0x7f6dbc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 --2- 172.21.15.27:0/1676601579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc10bad0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 --2- 172.21.15.27:0/1676601579 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc105d80 0x7f6dbc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 -- 172.21.15.27:0/1676601579 >> 172.21.15.27:0/1676601579 conn(0x7f6dbc101810 msgr2=0x7f6dbc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 -- 172.21.15.27:0/1676601579 shutdown_connections 2024-09-21T01:18:03.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.520+0000 7f6dc4200640 1 -- 172.21.15.27:0/1676601579 wait complete. 2024-09-21T01:18:03.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.521+0000 7f6dc4200640 1 Processor -- start 2024-09-21T01:18:03.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 -- start start 2024-09-21T01:18:03.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc105d80 0x7f6dbc19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:03.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:03.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10c010 0x7f6dbc1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc115d20 con 0x7f6dbc10c010 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc115ba0 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc4200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc115ea0 con 0x7f6dbc105d80 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc1800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc105d80 0x7f6dbc19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39036/0 (socket says 172.21.15.27:39036) 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc0e00640 1 -- 172.21.15.27:0/2059257771 learned_addr learned my addr 172.21.15.27:0/2059257771 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.522+0000 7f6dc2200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10c010 0x7f6dbc1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc0e00640 1 -- 172.21.15.27:0/2059257771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc105d80 msgr2=0x7f6dbc19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc0e00640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc105d80 0x7f6dbc19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc0e00640 1 -- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10c010 msgr2=0x7f6dbc1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc0e00640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10c010 0x7f6dbc1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc0e00640 1 -- 172.21.15.27:0/2059257771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc1a2640 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc1800640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc105d80 0x7f6dbc19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.523+0000 7f6dc0e00640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc19cc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6da8005ab0 tx=0x7f6da8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.524+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da803f070 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.524+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8031b50 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.524+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8035030 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.524+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc1a28d0 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.524+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc1a9090 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.525+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc107000 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.527+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f6da8042070 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.529+0000 7f6dba000640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6da40b65e0 0x7f6da40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:03.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.529+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7f6da81077e0 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.529+0000 7f6dc1800640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6da40b65e0 0x7f6da40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:03.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.530+0000 7f6dc1800640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6da40b65e0 0x7f6da40b8af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6db00095f0 tx=0x7f6db0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:03.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.530+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6da80d28e0 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.729+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6dbc10de00 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.730+0000 7f6dba000640 1 -- 172.21.15.27:0/2059257771 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f6da80d7820 con 0x7f6dbc10b6c0 2024-09-21T01:18:03.731 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6da40b65e0 msgr2=0x7f6da40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6da40b65e0 0x7f6da40b8af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6db00095f0 tx=0x7f6db0005f90 comp rx=0 tx=0).stop 2024-09-21T01:18:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 msgr2=0x7f6dbc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc19cc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6da8005ab0 tx=0x7f6da8003b90 comp rx=0 tx=0).stop 2024-09-21T01:18:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 shutdown_connections 2024-09-21T01:18:03.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6da40b65e0 0x7f6da40b8af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dbc10c010 0x7f6dbc1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6dbc10b6c0 0x7f6dbc19cc70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 --2- 172.21.15.27:0/2059257771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dbc105d80 0x7f6dbc19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 >> 172.21.15.27:0/2059257771 conn(0x7f6dbc101810 msgr2=0x7f6dbc10ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 shutdown_connections 2024-09-21T01:18:03.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:03.733+0000 7f6dc4200640 1 -- 172.21.15.27:0/2059257771 wait complete. 2024-09-21T01:18:03.932 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:03 smithi098 ceph-mon[34372]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:03.932 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:03 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2059257771' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:04.082 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:03 smithi027 ceph-mon[30375]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:04.082 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:03 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2059257771' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:04.229 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:18:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:18:04.001+0000 7f76ec352640 -1 osd.11 0 log_to_monitors true 2024-09-21T01:18:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:03 smithi089 ceph-mon[34486]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:03 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2059257771' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726881460,"num_in_osds":12,"osd_in_since":1726881463,"num_remapped_pgs":0} 2024-09-21T01:18:05.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:04 smithi098 ceph-mon[34372]: from='osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:18:05.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:04 smithi098 ceph-mon[34372]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:18:05.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:04 smithi027 ceph-mon[30375]: from='osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:18:05.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:04 smithi027 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:18:05.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:04 smithi089 ceph-mon[34486]: from='osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:18:05.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:04 smithi089 ceph-mon[34486]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T01:18:05.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:18:06.179 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:06.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:06.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T01:18:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T01:18:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:18:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:06.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T01:18:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T01:18:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:18:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:06.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.852+0000 7f0bb5c00640 1 -- 172.21.15.27:0/625713706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01092f0 msgr2=0x7f0bb0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.852+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/625713706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01092f0 0x7f0bb0109700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009900 tx=0x7f0b9802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 -- 172.21.15.27:0/625713706 shutdown_connections 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/625713706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bb0109c40 0x7f0bb0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/625713706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01092f0 0x7f0bb0109700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/625713706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01039b0 0x7f0bb0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:06.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 -- 172.21.15.27:0/625713706 >> 172.21.15.27:0/625713706 conn(0x7f0bb00ff400 msgr2=0x7f0bb0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 -- 172.21.15.27:0/625713706 shutdown_connections 2024-09-21T01:18:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.853+0000 7f0bb5c00640 1 -- 172.21.15.27:0/625713706 wait complete. 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.854+0000 7f0bb5c00640 1 Processor -- start 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.854+0000 7f0bb5c00640 1 -- start start 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.854+0000 7f0bb5c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 0x7f0bb019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.854+0000 7f0bb5c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01092f0 0x7f0bb019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bb5c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bb0109c40 0x7f0bb01a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bb5c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0113900 con 0x7f0bb01039b0 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bb5c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0113780 con 0x7f0bb01092f0 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bb5c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb0113a80 con 0x7f0bb0109c40 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 0x7f0bb019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 0x7f0bb019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46630/0 (socket says 172.21.15.27:46630) 2024-09-21T01:18:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 -- 172.21.15.27:0/3322874022 learned_addr learned my addr 172.21.15.27:0/3322874022 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bb5200640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bb0109c40 0x7f0bb01a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0ba7e00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01092f0 0x7f0bb019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 -- 172.21.15.27:0/3322874022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bb0109c40 msgr2=0x7f0bb01a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bb0109c40 0x7f0bb01a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 -- 172.21.15.27:0/3322874022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01092f0 msgr2=0x7f0bb019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01092f0 0x7f0bb019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0bafe00640 1 -- 172.21.15.27:0/3322874022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb01a2480 con 0x7f0bb01039b0 2024-09-21T01:18:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.855+0000 7f0ba7e00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01092f0 0x7f0bb019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.856+0000 7f0bafe00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 0x7f0bb019c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000e9f0 tx=0x7f0ba000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.856+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000ccc0 con 0x7f0bb01039b0 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.856+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba00108e0 con 0x7f0bb01039b0 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.856+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb01a2770 con 0x7f0bb01039b0 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.856+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0018880 con 0x7f0bb01039b0 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.856+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb01a8ed0 con 0x7f0bb01039b0 2024-09-21T01:18:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.857+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb0103dc0 con 0x7f0bb01039b0 2024-09-21T01:18:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.860+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0ba0010430 con 0x7f0bb01039b0 2024-09-21T01:18:06.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.861+0000 7f0bad600640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b940b6560 0x7f0b940b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:06.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.862+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7062+0+0 (secure 0 0 0) 0x7f0ba00e78a0 con 0x7f0bb01039b0 2024-09-21T01:18:06.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.862+0000 7f0ba7e00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b940b6560 0x7f0b940b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:06.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.863+0000 7f0ba7e00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b940b6560 0x7f0b940b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b98000c00 tx=0x7f0b98031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:06.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:06.868+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ba00ae570 con 0x7f0bb01039b0 2024-09-21T01:18:06.898 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:18:06.621+0000 7f76e82d4640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T01:18:06.898 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:18:06.634+0000 7f76e00ca640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T01:18:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.068+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0bb01a9200 con 0x7f0bb01039b0 2024-09-21T01:18:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.069+0000 7f0bad600640 1 -- 172.21.15.27:0/3322874022 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f0ba00b2950 con 0x7f0bb01039b0 2024-09-21T01:18:07.071 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b940b6560 msgr2=0x7f0b940b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b940b6560 0x7f0b940b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b98000c00 tx=0x7f0b98031400 comp rx=0 tx=0).stop 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 msgr2=0x7f0bb019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 0x7f0bb019c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000e9f0 tx=0x7f0ba000edf0 comp rx=0 tx=0).stop 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 shutdown_connections 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b940b6560 0x7f0b940b8a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bb0109c40 0x7f0bb01a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0bb01092f0 0x7f0bb019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 --2- 172.21.15.27:0/3322874022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bb01039b0 0x7f0bb019c510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 >> 172.21.15.27:0/3322874022 conn(0x7f0bb00ff400 msgr2=0x7f0bb01069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 shutdown_connections 2024-09-21T01:18:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:07.072+0000 7f0bb5c00640 1 -- 172.21.15.27:0/3322874022 wait complete. 2024-09-21T01:18:07.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: Detected new or changed devices on smithi098 2024-09-21T01:18:07.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T01:18:07.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:18:07.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T01:18:07.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:07.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:07.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:06 smithi027 ceph-mon[30375]: from='osd.11 ' entity='osd.11' 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: Detected new or changed devices on smithi098 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:06 smithi089 ceph-mon[34486]: from='osd.11 ' entity='osd.11' 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: Detected new or changed devices on smithi098 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi098 to 4272M 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: osdmap e70: 12 total, 11 up, 12 in 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:07.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:06 smithi098 ceph-mon[34372]: from='osd.11 ' entity='osd.11' 2024-09-21T01:18:07.826 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726881460,"num_in_osds":12,"osd_in_since":1726881463,"num_remapped_pgs":0} 2024-09-21T01:18:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: purged_snaps scrub starts 2024-09-21T01:18:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: purged_snaps scrub ok 2024-09-21T01:18:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3322874022' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:08.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] boot 2024-09-21T01:18:08.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T01:18:08.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: purged_snaps scrub starts 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: purged_snaps scrub ok 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3322874022' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] boot 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T01:18:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: purged_snaps scrub starts 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: purged_snaps scrub ok 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3322874022' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: osd.11 [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] boot 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T01:18:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T01:18:08.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T01:18:09.225 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.806+0000 7f4e9ec00640 1 -- 172.21.15.27:0/543992396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981039b0 msgr2=0x7f4e98103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.806+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/543992396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981039b0 0x7f4e98103dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009ca0 tx=0x7f4e8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 -- 172.21.15.27:0/543992396 shutdown_connections 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/543992396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e98109c40 0x7f4e98110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/543992396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e981092f0 0x7f4e98109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/543992396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981039b0 0x7f4e98103dc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 -- 172.21.15.27:0/543992396 >> 172.21.15.27:0/543992396 conn(0x7f4e980ff440 msgr2=0x7f4e98101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 -- 172.21.15.27:0/543992396 shutdown_connections 2024-09-21T01:18:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.807+0000 7f4e9ec00640 1 -- 172.21.15.27:0/543992396 wait complete. 2024-09-21T01:18:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 Processor -- start 2024-09-21T01:18:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 -- start start 2024-09-21T01:18:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 0x7f4e9819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981092f0 0x7f4e9819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e98109c40 0x7f4e981a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98113950 con 0x7f4e981092f0 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4e981137d0 con 0x7f4e98109c40 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.808+0000 7f4e9ec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98113ad0 con 0x7f4e981039b0 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9ce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981092f0 0x7f4e9819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 0x7f4e9819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 0x7f4e9819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:40032/0 (socket says 172.21.15.27:40032) 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 -- 172.21.15.27:0/2448168868 learned_addr learned my addr 172.21.15.27:0/2448168868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9e200640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e98109c40 0x7f4e981a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 -- 172.21.15.27:0/2448168868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e98109c40 msgr2=0x7f4e981a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e98109c40 0x7f4e981a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 -- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981092f0 msgr2=0x7f4e9819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981092f0 0x7f4e9819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9d800640 1 -- 172.21.15.27:0/2448168868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e981a2490 con 0x7f4e981039b0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.809+0000 7f4e9ce00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981092f0 0x7f4e9819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.810+0000 7f4e9d800640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 0x7f4e9819c520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c005e30 tx=0x7f4e8c031840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.810+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c003b70 con 0x7f4e981039b0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.810+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e981a2720 con 0x7f4e981039b0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.810+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8c003d10 con 0x7f4e981039b0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.810+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c02fa50 con 0x7f4e981039b0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.810+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e981a8ee0 con 0x7f4e981039b0 2024-09-21T01:18:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.811+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e98103dc0 con 0x7f4e981039b0 2024-09-21T01:18:09.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.814+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4e8c02fbf0 con 0x7f4e981039b0 2024-09-21T01:18:09.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.815+0000 7f4e96000640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4e800b6560 0x7f4e800b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:09.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.815+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f4e8c1083a0 con 0x7f4e981039b0 2024-09-21T01:18:09.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.815+0000 7f4e9ce00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4e800b6560 0x7f4e800b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:09.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.816+0000 7f4e9ce00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4e800b6560 0x7f4e800b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e98105a60 tx=0x7f4e84005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:09.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:09.817+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e8c0cef60 con 0x7f4e981039b0 2024-09-21T01:18:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.017+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4e981a9140 con 0x7f4e981039b0 2024-09-21T01:18:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.018+0000 7f4e96000640 1 -- 172.21.15.27:0/2448168868 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f4e8c0d3340 con 0x7f4e981039b0 2024-09-21T01:18:10.020 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.020+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4e800b6560 msgr2=0x7f4e800b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.020+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4e800b6560 0x7f4e800b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e98105a60 tx=0x7f4e84005f90 comp rx=0 tx=0).stop 2024-09-21T01:18:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.020+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 msgr2=0x7f4e9819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.020+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 0x7f4e9819c520 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c005e30 tx=0x7f4e8c031840 comp rx=0 tx=0).stop 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 shutdown_connections 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4e800b6560 0x7f4e800b8a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4e98109c40 0x7f4e981a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e981092f0 0x7f4e9819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 --2- 172.21.15.27:0/2448168868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e981039b0 0x7f4e9819c520 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 >> 172.21.15.27:0/2448168868 conn(0x7f4e980ff440 msgr2=0x7f4e981089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 shutdown_connections 2024-09-21T01:18:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:10.021+0000 7f4e9ec00640 1 -- 172.21.15.27:0/2448168868 wait complete. 2024-09-21T01:18:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:09 smithi089 ceph-mon[34486]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:09 smithi089 ceph-mon[34486]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T01:18:10.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:09 smithi027 ceph-mon[30375]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:10.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:09 smithi027 ceph-mon[30375]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T01:18:10.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:09 smithi098 ceph-mon[34372]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:09 smithi098 ceph-mon[34372]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T01:18:10.794 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726881487,"num_in_osds":12,"osd_in_since":1726881463,"num_remapped_pgs":0} 2024-09-21T01:18:10.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:18:11.176 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:11.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:10 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2448168868' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:11.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:10 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2448168868' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:11.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:10 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2448168868' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T01:18:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 -- 172.21.15.27:0/3287714652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 msgr2=0x7f323410d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 --2- 172.21.15.27:0/3287714652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323410d670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02a410 comp rx=0 tx=0).stop 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 -- 172.21.15.27:0/3287714652 shutdown_connections 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 --2- 172.21.15.27:0/3287714652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 0x7f3234110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 --2- 172.21.15.27:0/3287714652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323410d670 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 --2- 172.21.15.27:0/3287714652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 0x7f3234104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 -- 172.21.15.27:0/3287714652 >> 172.21.15.27:0/3287714652 conn(0x7f32340fe2d0 msgr2=0x7f3234100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 -- 172.21.15.27:0/3287714652 shutdown_connections 2024-09-21T01:18:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.814+0000 7f3239800640 1 -- 172.21.15.27:0/3287714652 wait complete. 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.815+0000 7f3239800640 1 Processor -- start 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.815+0000 7f3239800640 1 -- start start 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3239800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 0x7f323419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3239800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3239800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 0x7f32341a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3239800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3234113890 con 0x7f3234105120 2024-09-21T01:18:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3239800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3234113710 con 0x7f32341047d0 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3239800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3234113a10 con 0x7f323410dc30 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f322be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f322be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46672/0 (socket says 172.21.15.27:46672) 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3238e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 0x7f32341a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f3233e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 0x7f323419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.816+0000 7f322be00640 1 -- 172.21.15.27:0/1189797751 learned_addr learned my addr 172.21.15.27:0/1189797751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3233e00640 1 -- 172.21.15.27:0/1189797751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 msgr2=0x7f32341a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3233e00640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 0x7f32341a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3233e00640 1 -- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 msgr2=0x7f323419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3233e00640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3233e00640 1 -- 172.21.15.27:0/1189797751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32341a2430 con 0x7f32341047d0 2024-09-21T01:18:11.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3238e00640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 0x7f32341a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3233e00640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 0x7f323419c4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9f0 tx=0x7f322400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322400ccc0 con 0x7f32341047d0 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32240108e0 con 0x7f32341047d0 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32341a2720 con 0x7f32341047d0 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.817+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224018880 con 0x7f32341047d0 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.818+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32341a8e80 con 0x7f32341047d0 2024-09-21T01:18:11.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.818+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3234104be0 con 0x7f32341047d0 2024-09-21T01:18:11.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.821+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3224010430 con 0x7f32341047d0 2024-09-21T01:18:11.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.823+0000 7f3231600640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f32180b6560 0x7f32180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:11.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.823+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f32240e7800 con 0x7f32341047d0 2024-09-21T01:18:11.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.823+0000 7f322be00640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f32180b6560 0x7f32180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:11.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.823+0000 7f322be00640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f32180b6560 0x7f32180b8a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3234101df0 tx=0x7f321c009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:11.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:11.825+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f32240b27a0 con 0x7f32341047d0 2024-09-21T01:18:12.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.025+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32341a91b0 con 0x7f32341047d0 2024-09-21T01:18:12.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.028+0000 7f3231600640 1 -- 172.21.15.27:0/1189797751 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18723 (secure 0 0 0) 0x7f32240b76e0 con 0x7f32341047d0 2024-09-21T01:18:12.029 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:12.029 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":72,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:18:08.926984+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:18:12.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.029+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f32180b6560 msgr2=0x7f32180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:12.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.029+0000 7f3239800640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f32180b6560 0x7f32180b8a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3234101df0 tx=0x7f321c009560 comp rx=0 tx=0).stop 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.029+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 msgr2=0x7f323419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.029+0000 7f3239800640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 0x7f323419c4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9f0 tx=0x7f322400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 shutdown_connections 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f32180b6560 0x7f32180b8a70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f323410dc30 0x7f32341a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3234105120 0x7f323419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 --2- 172.21.15.27:0/1189797751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f32341047d0 0x7f323419c4c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 >> 172.21.15.27:0/1189797751 conn(0x7f32340fe2d0 msgr2=0x7f32340fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 shutdown_connections 2024-09-21T01:18:12.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:12.030+0000 7f3239800640 1 -- 172.21.15.27:0/1189797751 wait complete. 2024-09-21T01:18:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:11 smithi089 ceph-mon[34486]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:12.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:11 smithi098 ceph-mon[34372]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:12.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:11 smithi027 ceph-mon[30375]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:12.962 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:13:52.790954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T01:18:12.963 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T01:18:13.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:12 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1189797751' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:18:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:12 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1189797751' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:18:13.299 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:12 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1189797751' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:18:13.342 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 -- 172.21.15.27:0/1651918187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150114740 msgr2=0x7f8150116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:14.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 --2- 172.21.15.27:0/1651918187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150114740 0x7f8150116b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8148009ca0 tx=0x7f814802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 -- 172.21.15.27:0/1651918187 shutdown_connections 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 --2- 172.21.15.27:0/1651918187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150114740 0x7f8150116b60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 --2- 172.21.15.27:0/1651918187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8150078290 0x7f8150114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 --2- 172.21.15.27:0/1651918187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150077940 0x7f8150077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 -- 172.21.15.27:0/1651918187 >> 172.21.15.27:0/1651918187 conn(0x7f8150101810 msgr2=0x7f8150103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 -- 172.21.15.27:0/1651918187 shutdown_connections 2024-09-21T01:18:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.121+0000 7f8156200640 1 -- 172.21.15.27:0/1651918187 wait complete. 2024-09-21T01:18:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.122+0000 7f8156200640 1 Processor -- start 2024-09-21T01:18:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 -- start start 2024-09-21T01:18:14.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 0x7f81501a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8150078290 0x7f81501a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150114740 0x7f81501a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f815011a0d0 con 0x7f8150114740 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8150119f50 con 0x7f8150078290 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8156200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f815011a250 con 0x7f8150077940 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f814f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 0x7f81501a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f814f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 0x7f81501a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:40058/0 (socket says 172.21.15.27:40058) 2024-09-21T01:18:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f814f400640 1 -- 172.21.15.27:0/798685521 learned_addr learned my addr 172.21.15.27:0/798685521 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f8146a00640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8150078290 0x7f81501a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.123+0000 7f814fe00640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150114740 0x7f81501a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814f400640 1 -- 172.21.15.27:0/798685521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8150078290 msgr2=0x7f81501a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814f400640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8150078290 0x7f81501a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814f400640 1 -- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150114740 msgr2=0x7f81501a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814f400640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150114740 0x7f81501a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814f400640 1 -- 172.21.15.27:0/798685521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81501a69d0 con 0x7f8150077940 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814fe00640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150114740 0x7f81501a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:18:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814f400640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 0x7f81501a0a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f814000e9f0 tx=0x7f814000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814000cd80 con 0x7f8150077940 2024-09-21T01:18:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81400108e0 con 0x7f8150077940 2024-09-21T01:18:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81501a6cc0 con 0x7f8150077940 2024-09-21T01:18:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140018980 con 0x7f8150077940 2024-09-21T01:18:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.124+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501ad420 con 0x7f8150077940 2024-09-21T01:18:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.125+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8150077d50 con 0x7f8150077940 2024-09-21T01:18:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.128+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8140010430 con 0x7f8150077940 2024-09-21T01:18:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.129+0000 7f814cc00640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f81340b65e0 0x7f81340b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.129+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f81400e73a0 con 0x7f8150077940 2024-09-21T01:18:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.129+0000 7f8146a00640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f81340b65e0 0x7f81340b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.130+0000 7f8146a00640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f81340b65e0 0x7f81340b8af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f815007b260 tx=0x7f813800b390 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.133+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81400adf60 con 0x7f8150077940 2024-09-21T01:18:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:13 smithi089 ceph-mon[34486]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:14.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.331+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f815002e640 con 0x7f8150077940 2024-09-21T01:18:14.333 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:13 smithi027 ceph-mon[30375]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:14.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.332+0000 7f814cc00640 1 -- 172.21.15.27:0/798685521 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f81400b2340 con 0x7f8150077940 2024-09-21T01:18:14.333 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T01:18:14.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.335+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f81340b65e0 msgr2=0x7f81340b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:14.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.335+0000 7f8156200640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f81340b65e0 0x7f81340b8af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f815007b260 tx=0x7f813800b390 comp rx=0 tx=0).stop 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 msgr2=0x7f81501a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 0x7f81501a0a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f814000e9f0 tx=0x7f814000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 shutdown_connections 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f81340b65e0 0x7f81340b8af0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8150114740 0x7f81501a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8150078290 0x7f81501a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 --2- 172.21.15.27:0/798685521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8150077940 0x7f81501a0a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 >> 172.21.15.27:0/798685521 conn(0x7f8150101810 msgr2=0x7f8150116750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 shutdown_connections 2024-09-21T01:18:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:14.336+0000 7f8156200640 1 -- 172.21.15.27:0/798685521 wait complete. 2024-09-21T01:18:14.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:13 smithi098 ceph-mon[34372]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:15.089 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:14 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/798685521' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:18:15.127 INFO:tasks.cephadm:Adding mds.a on smithi027 2024-09-21T01:18:15.127 INFO:tasks.cephadm:Adding mds.d on smithi027 2024-09-21T01:18:15.127 INFO:tasks.cephadm:Adding mds.g on smithi027 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.j on smithi027 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.b on smithi089 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.e on smithi089 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.h on smithi089 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.k on smithi089 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.c on smithi098 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.f on smithi098 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.i on smithi098 2024-09-21T01:18:15.128 INFO:tasks.cephadm:Adding mds.l on smithi098 2024-09-21T01:18:15.128 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l' 2024-09-21T01:18:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:14 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/798685521' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:18:15.376 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:18:15.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:14 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/798685521' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:18:15.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 -- 172.21.15.98:0/1027958064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f4102a60 msgr2=0x7f04f4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:15.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/1027958064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f4102a60 0x7f04f4102e70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009900 tx=0x7f04e402a410 comp rx=0 tx=0).stop 2024-09-21T01:18:15.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 -- 172.21.15.98:0/1027958064 shutdown_connections 2024-09-21T01:18:15.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/1027958064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04f4108cf0 0x7f04f410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:15.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/1027958064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f41083a0 0x7f04f41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:15.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/1027958064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f4102a60 0x7f04f4102e70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 -- 172.21.15.98:0/1027958064 >> 172.21.15.98:0/1027958064 conn(0x7f04f4079730 msgr2=0x7f04f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 -- 172.21.15.98:0/1027958064 shutdown_connections 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.874+0000 7f04f8f1a640 1 -- 172.21.15.98:0/1027958064 wait complete. 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 Processor -- start 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 -- start start 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4102a60 0x7f04f4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 0x7f04f41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04f4108cf0 0x7f04f419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4112960 con 0x7f04f4108cf0 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41127e0 con 0x7f04f4102a60 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f8f1a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4112ae0 con 0x7f04f41083a0 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 0x7f04f41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:15.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 0x7f04f41999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40578/0 (socket says 172.21.15.98:40578) 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.875+0000 7f04f1d9b640 1 -- 172.21.15.98:0/3903315147 learned_addr learned my addr 172.21.15.98:0/3903315147 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f259c640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4102a60 0x7f04f4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f1d9b640 1 -- 172.21.15.98:0/3903315147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4102a60 msgr2=0x7f04f4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f1d9b640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4102a60 0x7f04f4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f1d9b640 1 -- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04f4108cf0 msgr2=0x7f04f419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f1d9b640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04f4108cf0 0x7f04f419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f1d9b640 1 -- 172.21.15.98:0/3903315147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f419f2c0 con 0x7f04f41083a0 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f1d9b640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 0x7f04f41999b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009830 tx=0x7f04dc009cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc005020 con 0x7f04f41083a0 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04dc0047a0 con 0x7f04f41083a0 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f419f5b0 con 0x7f04f41083a0 2024-09-21T01:18:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.876+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007b20 con 0x7f04f41083a0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.877+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f406ae60 con 0x7f04f41083a0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.878+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f4103ce0 con 0x7f04f41083a0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.879+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f04dc004950 con 0x7f04f41083a0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.880+0000 7f04eb7fe640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f04e00b6610 0x7f04e00b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.880+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f04dc0e8ba0 con 0x7f04f41083a0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.880+0000 7f04f259c640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f04e00b6610 0x7f04e00b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.881+0000 7f04f259c640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f04e00b6610 0x7f04e00b8b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f04e4031440 tx=0x7f04e4009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:15.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:15.883+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04dc0af800 con 0x7f04f41083a0 2024-09-21T01:18:16.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.076+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}) -- 0x7f04f406b180 con 0x7f04e00b6610 2024-09-21T01:18:16.077 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:15 smithi098 ceph-mon[34372]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:16.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.084+0000 7f04eb7fe640 1 -- 172.21.15.98:0/3903315147 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f04f406b180 con 0x7f04e00b6610 2024-09-21T01:18:16.085 INFO:teuthology.orchestra.run.smithi098.stdout:Scheduled mds.all update... 2024-09-21T01:18:16.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f04e00b6610 msgr2=0x7f04e00b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:16.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f04e00b6610 0x7f04e00b8b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f04e4031440 tx=0x7f04e4009560 comp rx=0 tx=0).stop 2024-09-21T01:18:16.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 msgr2=0x7f04f41999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 0x7f04f41999b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009830 tx=0x7f04dc009cf0 comp rx=0 tx=0).stop 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 shutdown_connections 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f04e00b6610 0x7f04e00b8b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04f4108cf0 0x7f04f419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04f41083a0 0x7f04f41999b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 --2- 172.21.15.98:0/3903315147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04f4102a60 0x7f04f4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.086+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 >> 172.21.15.98:0/3903315147 conn(0x7f04f4079730 msgr2=0x7f04f41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.087+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 shutdown_connections 2024-09-21T01:18:16.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-21T01:18:16.087+0000 7f04f8f1a640 1 -- 172.21.15.98:0/3903315147 wait complete. 2024-09-21T01:18:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:15 smithi089 ceph-mon[34486]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:15 smithi027 ceph-mon[30375]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:16.809 DEBUG:teuthology.orchestra.run.smithi027:mds.a> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T01:18:16.813 DEBUG:teuthology.orchestra.run.smithi027:mds.d> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T01:18:16.858 DEBUG:teuthology.orchestra.run.smithi027:mds.g> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T01:18:16.903 DEBUG:teuthology.orchestra.run.smithi027:mds.j> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T01:18:16.948 DEBUG:teuthology.orchestra.run.smithi089:mds.b> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T01:18:16.950 DEBUG:teuthology.orchestra.run.smithi089:mds.e> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T01:18:16.994 DEBUG:teuthology.orchestra.run.smithi089:mds.h> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T01:18:17.036 DEBUG:teuthology.orchestra.run.smithi089:mds.k> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T01:18:17.079 DEBUG:teuthology.orchestra.run.smithi098:mds.c> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T01:18:17.082 DEBUG:teuthology.orchestra.run.smithi098:mds.f> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T01:18:17.125 DEBUG:teuthology.orchestra.run.smithi098:mds.i> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T01:18:17.168 DEBUG:teuthology.orchestra.run.smithi098:mds.l> sudo journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='client.24649 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:17.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:17.211 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T01:18:17.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='client.24649 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='client.24649 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:18:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: Saving service mds.all spec with placement smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T01:18:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:17.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:17.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:17.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:17.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:17.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:17.592 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:18.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:18 smithi098 ceph-mon[34372]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:18 smithi098 ceph-mon[34372]: Deploying daemon mds.l on smithi098 2024-09-21T01:18:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 -- 172.21.15.27:0/858360895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798109c40 msgr2=0x7f3798110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 --2- 172.21.15.27:0/858360895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798109c40 0x7f3798110470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3790009ca0 tx=0x7f379002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 -- 172.21.15.27:0/858360895 shutdown_connections 2024-09-21T01:18:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 --2- 172.21.15.27:0/858360895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3798109c40 0x7f3798110470 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 --2- 172.21.15.27:0/858360895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37981092f0 0x7f3798109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 --2- 172.21.15.27:0/858360895 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f3798103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 -- 172.21.15.27:0/858360895 >> 172.21.15.27:0/858360895 conn(0x7f37980ff400 msgr2=0x7f3798101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 -- 172.21.15.27:0/858360895 shutdown_connections 2024-09-21T01:18:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.185+0000 7f3797200640 1 -- 172.21.15.27:0/858360895 wait complete. 2024-09-21T01:18:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.186+0000 7f3797200640 1 Processor -- start 2024-09-21T01:18:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.186+0000 7f3797200640 1 -- start start 2024-09-21T01:18:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3797200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f379819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3797200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37981092f0 0x7f379819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3797200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3798109c40 0x7f379819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3797200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3798113950 con 0x7f37981092f0 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3797200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37981137d0 con 0x7f37981039b0 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3797200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3798113ad0 con 0x7f3798109c40 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3795400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37981092f0 0x7f379819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3795e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f379819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3795e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f379819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56082/0 (socket says 172.21.15.27:56082) 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3795e00640 1 -- 172.21.15.27:0/1773057402 learned_addr learned my addr 172.21.15.27:0/1773057402 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.187+0000 7f3796800640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3798109c40 0x7f379819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795e00640 1 -- 172.21.15.27:0/1773057402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3798109c40 msgr2=0x7f379819faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795e00640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3798109c40 0x7f379819faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795e00640 1 -- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37981092f0 msgr2=0x7f379819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:18.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795e00640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37981092f0 0x7f379819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795e00640 1 -- 172.21.15.27:0/1773057402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37981a01d0 con 0x7f37981039b0 2024-09-21T01:18:18.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795400640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37981092f0 0x7f379819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3795e00640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f379819a260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f378800e9f0 tx=0x7f378800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378800cd80 con 0x7f37981039b0 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37880108e0 con 0x7f37981039b0 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37981a04c0 con 0x7f37981039b0 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788018980 con 0x7f37981039b0 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.188+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37981a6c20 con 0x7f37981039b0 2024-09-21T01:18:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.189+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3798103dc0 con 0x7f37981039b0 2024-09-21T01:18:18.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.192+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3788010430 con 0x7f37981039b0 2024-09-21T01:18:18.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.193+0000 7f378ea00640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f377c0b6560 0x7f377c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:18.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.194+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f37880e77e0 con 0x7f37981039b0 2024-09-21T01:18:18.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.194+0000 7f3795400640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f377c0b6560 0x7f377c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:18.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.194+0000 7f3795400640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f377c0b6560 0x7f377c0b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3798106d30 tx=0x7f3784005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:18.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.195+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37880b2780 con 0x7f37981039b0 2024-09-21T01:18:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:18 smithi027 ceph-mon[30375]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:18 smithi027 ceph-mon[30375]: Deploying daemon mds.l on smithi098 2024-09-21T01:18:18.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.433+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f379819b390 con 0x7f37981039b0 2024-09-21T01:18:18.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.433+0000 7f378ea00640 1 -- 172.21.15.27:0/1773057402 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f37880b76c0 con 0x7f37981039b0 2024-09-21T01:18:18.434 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":1,"btime":"2024-09-21T01:09:34:736922+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T01:18:18.435 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 1 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f377c0b6560 msgr2=0x7f377c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f377c0b6560 0x7f377c0b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3798106d30 tx=0x7f3784005f90 comp rx=0 tx=0).stop 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 msgr2=0x7f379819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f379819a260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f378800e9f0 tx=0x7f378800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 shutdown_connections 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f377c0b6560 0x7f377c0b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3798109c40 0x7f379819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37981092f0 0x7f379819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 --2- 172.21.15.27:0/1773057402 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37981039b0 0x7f379819a260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 >> 172.21.15.27:0/1773057402 conn(0x7f37980ff400 msgr2=0x7f3798101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 shutdown_connections 2024-09-21T01:18:18.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:18.436+0000 7f3797200640 1 -- 172.21.15.27:0/1773057402 wait complete. 2024-09-21T01:18:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:18 smithi089 ceph-mon[34486]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:18 smithi089 ceph-mon[34486]: Deploying daemon mds.l on smithi098 2024-09-21T01:18:18.838 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:18 smithi098 systemd[1]: Starting Ceph mds.l for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:19.099 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:18 smithi098 podman[63251]: 2024-09-21 01:18:18.740661648 +0000 UTC m=+0.015754328 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:19.099 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:18 smithi098 podman[63251]: 2024-09-21 01:18:18.874059088 +0000 UTC m=+0.149151809 container create 72874aa4d923458bad0bb6be5bd948c8262010e3d4b77044f61d7f6f7c1bf80f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:19 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1773057402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:19 smithi098 podman[63251]: 2024-09-21 01:18:19.152358474 +0000 UTC m=+0.427451149 container init 72874aa4d923458bad0bb6be5bd948c8262010e3d4b77044f61d7f6f7c1bf80f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:19 smithi098 podman[63251]: 2024-09-21 01:18:19.15533789 +0000 UTC m=+0.430430568 container start 72874aa4d923458bad0bb6be5bd948c8262010e3d4b77044f61d7f6f7c1bf80f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:19 smithi098 ceph-mds[63270]: -- 172.21.15.98:0/1588281347 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587154e5860 con 0x55871968da80 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l[63266]: starting mds.l at 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:19 smithi098 bash[63251]: 72874aa4d923458bad0bb6be5bd948c8262010e3d4b77044f61d7f6f7c1bf80f 2024-09-21T01:18:19.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:18:19 smithi098 systemd[1]: Started Ceph mds.l for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:19 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1773057402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:19.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:19 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1773057402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6832/2535514726,v1:172.21.15.98:6833/2535514726] up:boot 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: fsmap 1 up:standby 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:20.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:20.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6832/2535514726,v1:172.21.15.98:6833/2535514726] up:boot 2024-09-21T01:18:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: fsmap 1 up:standby 2024-09-21T01:18:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T01:18:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:20.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:20.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6832/2535514726,v1:172.21.15.98:6833/2535514726] up:boot 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: fsmap 1 up:standby 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:21.207 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:21.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:21 smithi098 ceph-mon[34372]: Deploying daemon mds.k on smithi089 2024-09-21T01:18:21.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:21 smithi089 ceph-mon[34486]: Deploying daemon mds.k on smithi089 2024-09-21T01:18:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:21 smithi027 ceph-mon[30375]: Deploying daemon mds.k on smithi089 2024-09-21T01:18:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:21.750 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 podman[62563]: 2024-09-21 01:18:21.638029135 +0000 UTC m=+0.135877110 container create 8e78bce89c243b91d89e67e372106ddfd00fd80a989acc805547080d1e0d83c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T01:18:22.117 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 podman[62563]: 2024-09-21 01:18:21.881798795 +0000 UTC m=+0.379646761 container init 8e78bce89c243b91d89e67e372106ddfd00fd80a989acc805547080d1e0d83c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:18:22.117 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 podman[62563]: 2024-09-21 01:18:21.884483022 +0000 UTC m=+0.382330987 container start 8e78bce89c243b91d89e67e372106ddfd00fd80a989acc805547080d1e0d83c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T01:18:22.118 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 ceph-mds[62582]: -- 172.21.15.89:0/1521908782 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4ab16bc20 con 0x55f4af313a80 2024-09-21T01:18:22.118 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k[62578]: starting mds.k at 2024-09-21T01:18:22.118 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 bash[62563]: 8e78bce89c243b91d89e67e372106ddfd00fd80a989acc805547080d1e0d83c3 2024-09-21T01:18:22.118 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:18:21 smithi089 systemd[1]: Started Ceph mds.k for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:22.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 -- 172.21.15.27:0/447912405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 msgr2=0x7f8f78103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/447912405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 0x7f8f78103f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009900 tx=0x7f8f6802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 -- 172.21.15.27:0/447912405 shutdown_connections 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/447912405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f78110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/447912405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 0x7f8f781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/447912405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 0x7f8f78103f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 -- 172.21.15.27:0/447912405 >> 172.21.15.27:0/447912405 conn(0x7f8f7807b850 msgr2=0x7f8f7807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 -- 172.21.15.27:0/447912405 shutdown_connections 2024-09-21T01:18:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.232+0000 7f8f7ec00640 1 -- 172.21.15.27:0/447912405 wait complete. 2024-09-21T01:18:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.233+0000 7f8f7ec00640 1 Processor -- start 2024-09-21T01:18:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.233+0000 7f8f7ec00640 1 -- start start 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7ec00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 0x7f8f7819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7ec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 0x7f8f7819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f7819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7ec00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78113b00 con 0x7f8f78103b60 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78113980 con 0x7f8f78109df0 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7ec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78113c80 con 0x7f8f781094a0 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 0x7f8f7819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f7819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f7819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:56096/0 (socket says 172.21.15.27:56096) 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f7cc00640 1 -- 172.21.15.27:0/3401542316 learned_addr learned my addr 172.21.15.27:0/3401542316 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.234+0000 7f8f77400640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 0x7f8f7819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f77400640 1 -- 172.21.15.27:0/3401542316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 msgr2=0x7f8f7819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f77400640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f7819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f77400640 1 -- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 msgr2=0x7f8f7819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f77400640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 0x7f8f7819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f77400640 1 -- 172.21.15.27:0/3401542316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f781a04e0 con 0x7f8f781094a0 2024-09-21T01:18:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f7cc00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f7819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f77400640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 0x7f8f7819ab10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806c9f0 tx=0x7f8f6400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400cc40 con 0x7f8f781094a0 2024-09-21T01:18:22.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f781a07d0 con 0x7f8f781094a0 2024-09-21T01:18:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f6400cde0 con 0x7f8f781094a0 2024-09-21T01:18:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.235+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64004a00 con 0x7f8f781094a0 2024-09-21T01:18:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.236+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f780755f0 con 0x7f8f781094a0 2024-09-21T01:18:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.237+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f78104de0 con 0x7f8f781094a0 2024-09-21T01:18:22.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.239+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8f64004ba0 con 0x7f8f781094a0 2024-09-21T01:18:22.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.240+0000 7f8f74c00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8f5c0b6560 0x7f8f5c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:22.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.241+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f8f640e7360 con 0x7f8f781094a0 2024-09-21T01:18:22.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.241+0000 7f8f77e00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8f5c0b6560 0x7f8f5c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:22.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.241+0000 7f8f77e00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8f5c0b6560 0x7f8f5c0b8a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005ab0 tx=0x7f8f68005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:22.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.242+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f640adf20 con 0x7f8f781094a0 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.494+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f7810bbc0 con 0x7f8f781094a0 2024-09-21T01:18:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.495+0000 7f8f74c00640 1 -- 172.21.15.27:0/3401542316 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2254 (secure 0 0 0) 0x7f8f640b2300 con 0x7f8f781094a0 2024-09-21T01:18:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:22.496 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"btime":"2024-09-21T01:18:22:107258+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T01:18:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8f5c0b6560 msgr2=0x7f8f5c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8f5c0b6560 0x7f8f5c0b8a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8f68005ab0 tx=0x7f8f68005a20 comp rx=0 tx=0).stop 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 msgr2=0x7f8f7819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 0x7f8f7819ab10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806c9f0 tx=0x7f8f6400ee50 comp rx=0 tx=0).stop 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 shutdown_connections 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8f5c0b6560 0x7f8f5c0b8a70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8f78109df0 0x7f8f7819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f781094a0 0x7f8f7819ab10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 --2- 172.21.15.27:0/3401542316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f78103b60 0x7f8f7819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 >> 172.21.15.27:0/3401542316 conn(0x7f8f7807b850 msgr2=0x7f8f781075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:22.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 shutdown_connections 2024-09-21T01:18:22.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:22.498+0000 7f8f7ec00640 1 -- 172.21.15.27:0/3401542316 wait complete. 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:23.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:23 smithi089 ceph-mon[34486]: Deploying daemon mds.j on smithi027 2024-09-21T01:18:23.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:23 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6832/39744273,v1:172.21.15.89:6833/39744273] up:boot 2024-09-21T01:18:23.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:23 smithi089 ceph-mon[34486]: fsmap 2 up:standby 2024-09-21T01:18:23.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:18:23.383 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:23 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3401542316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:23.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:23 smithi098 ceph-mon[34372]: Deploying daemon mds.j on smithi027 2024-09-21T01:18:23.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:23 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6832/39744273,v1:172.21.15.89:6833/39744273] up:boot 2024-09-21T01:18:23.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:23 smithi098 ceph-mon[34372]: fsmap 2 up:standby 2024-09-21T01:18:23.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:18:23.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:23 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3401542316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:23.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:23 smithi027 ceph-mon[30375]: Deploying daemon mds.j on smithi027 2024-09-21T01:18:23.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:23 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6832/39744273,v1:172.21.15.89:6833/39744273] up:boot 2024-09-21T01:18:23.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:23 smithi027 ceph-mon[30375]: fsmap 2 up:standby 2024-09-21T01:18:23.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:18:23.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:23 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3401542316' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:24 smithi098 ceph-mon[34372]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:24 smithi027 ceph-mon[30375]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:24.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:24 smithi089 ceph-mon[34486]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:25.562 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 systemd[1]: Starting Ceph mds.j for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:25.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:25.813 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 podman[71927]: 2024-09-21 01:18:25.466313831 +0000 UTC m=+0.027529578 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:25.813 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 podman[71927]: 2024-09-21 01:18:25.585878836 +0000 UTC m=+0.147094566 container create 612eec67dae253f57a8f13bea185d3941836dc7fc4f808d676f0f17e0cec7726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T01:18:25.814 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 podman[71927]: 2024-09-21 01:18:25.813400785 +0000 UTC m=+0.374616542 container init 612eec67dae253f57a8f13bea185d3941836dc7fc4f808d676f0f17e0cec7726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T01:18:26.135 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:26.136 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 podman[71927]: 2024-09-21 01:18:25.822259031 +0000 UTC m=+0.383474762 container start 612eec67dae253f57a8f13bea185d3941836dc7fc4f808d676f0f17e0cec7726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:18:26.136 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 ceph-mds[71964]: -- 172.21.15.27:0/527071112 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d1602d860 con 0x558d1a1d5600 2024-09-21T01:18:26.136 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j[71942]: starting mds.j at 2024-09-21T01:18:26.136 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 bash[71927]: 612eec67dae253f57a8f13bea185d3941836dc7fc4f808d676f0f17e0cec7726 2024-09-21T01:18:26.136 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:18:25 smithi027 systemd[1]: Started Ceph mds.j for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:26.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:26.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 -- 172.21.15.27:0/869234006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f01094c0 msgr2=0x7fa9f01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:26.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 --2- 172.21.15.27:0/869234006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f01094c0 0x7fa9f01098d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009900 tx=0x7fa9dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:26.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 -- 172.21.15.27:0/869234006 shutdown_connections 2024-09-21T01:18:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 --2- 172.21.15.27:0/869234006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f0109e10 0x7fa9f0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 --2- 172.21.15.27:0/869234006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f01094c0 0x7fa9f01098d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 --2- 172.21.15.27:0/869234006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 0x7fa9f0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 -- 172.21.15.27:0/869234006 >> 172.21.15.27:0/869234006 conn(0x7fa9f00ff5d0 msgr2=0x7fa9f0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 -- 172.21.15.27:0/869234006 shutdown_connections 2024-09-21T01:18:26.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.797+0000 7fa9f7000640 1 -- 172.21.15.27:0/869234006 wait complete. 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.798+0000 7fa9f7000640 1 Processor -- start 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.798+0000 7fa9f7000640 1 -- start start 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f7000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 0x7fa9f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f7000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 0x7fa9f0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f7000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f0109e10 0x7fa9f0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f7000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0113b40 con 0x7fa9f0109e10 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f7000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01139c0 con 0x7fa9f01094c0 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f7000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0113cc0 con 0x7fa9f0103b80 2024-09-21T01:18:26.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 0x7fa9f0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9f5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f0109e10 0x7fa9f0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 0x7fa9f0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45724/0 (socket says 172.21.15.27:45724) 2024-09-21T01:18:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9ef400640 1 -- 172.21.15.27:0/2095511228 learned_addr learned my addr 172.21.15.27:0/2095511228 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:26.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.799+0000 7fa9efe00640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 0x7fa9f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ef400640 1 -- 172.21.15.27:0/2095511228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 msgr2=0x7fa9f0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ef400640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 0x7fa9f0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ef400640 1 -- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f0109e10 msgr2=0x7fa9f0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ef400640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f0109e10 0x7fa9f0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ef400640 1 -- 172.21.15.27:0/2095511228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f01a87d0 con 0x7fa9f01094c0 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9efe00640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 0x7fa9f0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ef400640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 0x7fa9f0075c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005ab0 tx=0x7fa9dc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.800+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc03f070 con 0x7fa9f01094c0 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.801+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9dc031b50 con 0x7fa9f01094c0 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.801+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f01a8a60 con 0x7fa9f01094c0 2024-09-21T01:18:26.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.801+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc035030 con 0x7fa9f01094c0 2024-09-21T01:18:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.801+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f01a9010 con 0x7fa9f01094c0 2024-09-21T01:18:26.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.802+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f0103f90 con 0x7fa9f01094c0 2024-09-21T01:18:26.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.805+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fa9dc042070 con 0x7fa9f01094c0 2024-09-21T01:18:26.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.807+0000 7fa9ecc00640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9d40b65e0 0x7fa9d40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:26.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.807+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa9dc107df0 con 0x7fa9f01094c0 2024-09-21T01:18:26.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.807+0000 7fa9efe00640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9d40b65e0 0x7fa9d40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:26.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.808+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9dc0d2d90 con 0x7fa9f01094c0 2024-09-21T01:18:26.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:26.808+0000 7fa9efe00640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9d40b65e0 0x7fa9d40b8af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0077160 tx=0x7fa9e0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.064+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9f002e640 con 0x7fa9f01094c0 2024-09-21T01:18:27.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.065+0000 7fa9ecc00640 1 -- 172.21.15.27:0/2095511228 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3079 (secure 0 0 0) 0x7fa9dc0d7cd0 con 0x7fa9f01094c0 2024-09-21T01:18:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:27.066 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":4,"btime":"2024-09-21T01:18:26:129745+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-09-21T01:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 4 2024-09-21T01:18:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.067+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9d40b65e0 msgr2=0x7fa9d40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.067+0000 7fa9f7000640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9d40b65e0 0x7fa9d40b8af0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0077160 tx=0x7fa9e0005f90 comp rx=0 tx=0).stop 2024-09-21T01:18:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.067+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 msgr2=0x7fa9f0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.067+0000 7fa9f7000640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 0x7fa9f0075c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc005ab0 tx=0x7fa9dc003b90 comp rx=0 tx=0).stop 2024-09-21T01:18:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 shutdown_connections 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9d40b65e0 0x7fa9d40b8af0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9f0109e10 0x7fa9f0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9f01094c0 0x7fa9f0075c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 --2- 172.21.15.27:0/2095511228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9f0103b80 0x7fa9f0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 >> 172.21.15.27:0/2095511228 conn(0x7fa9f00ff5d0 msgr2=0x7fa9f01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 shutdown_connections 2024-09-21T01:18:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:27.068+0000 7fa9f7000640 1 -- 172.21.15.27:0/2095511228 wait complete. 2024-09-21T01:18:27.390 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:27 smithi027 ceph-mon[30375]: Deploying daemon mds.d on smithi027 2024-09-21T01:18:27.390 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:27 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6834/2864798916,v1:172.21.15.27:6835/2864798916] up:boot 2024-09-21T01:18:27.390 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:27 smithi027 ceph-mon[30375]: fsmap 3 up:standby 2024-09-21T01:18:27.390 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:27 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:18:27.390 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:27 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2095511228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:27 smithi098 ceph-mon[34372]: Deploying daemon mds.d on smithi027 2024-09-21T01:18:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:27 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6834/2864798916,v1:172.21.15.27:6835/2864798916] up:boot 2024-09-21T01:18:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:27 smithi098 ceph-mon[34372]: fsmap 3 up:standby 2024-09-21T01:18:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:27 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:18:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:27 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2095511228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:27 smithi089 ceph-mon[34486]: Deploying daemon mds.d on smithi027 2024-09-21T01:18:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:27 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6834/2864798916,v1:172.21.15.27:6835/2864798916] up:boot 2024-09-21T01:18:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:27 smithi089 ceph-mon[34486]: fsmap 3 up:standby 2024-09-21T01:18:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:27 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:18:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:27 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2095511228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:28 smithi027 ceph-mon[30375]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:28 smithi098 ceph-mon[34372]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:28.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:28 smithi089 ceph-mon[34486]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:29.666 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:29 smithi027 systemd[1]: Starting Ceph mds.d for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:29.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:29 smithi027 podman[72307]: 2024-09-21 01:18:29.569942405 +0000 UTC m=+0.028304148 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:29.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:29 smithi027 podman[72307]: 2024-09-21 01:18:29.708774606 +0000 UTC m=+0.167136348 container create 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T01:18:30.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:30 smithi098 ceph-mon[34372]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:30.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:30 smithi027 ceph-mon[30375]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:30.426 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:30 smithi027 podman[72307]: 2024-09-21 01:18:30.015381609 +0000 UTC m=+0.473743351 container init 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:18:30.426 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:30 smithi027 podman[72307]: 2024-09-21 01:18:30.022290781 +0000 UTC m=+0.480652523 container start 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T01:18:30.426 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:30 smithi027 bash[72307]: 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 2024-09-21T01:18:30.426 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:30 smithi027 systemd[1]: Started Ceph mds.d for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:30.426 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:30 smithi027 ceph-mds[72326]: -- 172.21.15.27:0/2588206036 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbc9f33860 con 0x55fbce0db180 2024-09-21T01:18:30.426 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:18:30 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d[72322]: starting mds.d at 2024-09-21T01:18:30.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:30 smithi089 ceph-mon[34486]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:30.762 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 -- 172.21.15.27:0/2393600631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 msgr2=0x7f8a24103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 --2- 172.21.15.27:0/2393600631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 0x7f8a24103da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009860 tx=0x7f8a1402a340 comp rx=0 tx=0).stop 2024-09-21T01:18:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 -- 172.21.15.27:0/2393600631 shutdown_connections 2024-09-21T01:18:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 --2- 172.21.15.27:0/2393600631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a24109c20 0x7f8a24110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 --2- 172.21.15.27:0/2393600631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a241092d0 0x7f8a241096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 --2- 172.21.15.27:0/2393600631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 0x7f8a24103da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 -- 172.21.15.27:0/2393600631 >> 172.21.15.27:0/2393600631 conn(0x7f8a240ff3e0 msgr2=0x7f8a24101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 -- 172.21.15.27:0/2393600631 shutdown_connections 2024-09-21T01:18:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.397+0000 7f8a29600640 1 -- 172.21.15.27:0/2393600631 wait complete. 2024-09-21T01:18:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.398+0000 7f8a29600640 1 Processor -- start 2024-09-21T01:18:31.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.398+0000 7f8a29600640 1 -- start start 2024-09-21T01:18:31.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.398+0000 7f8a29600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 0x7f8a2419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:31.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.398+0000 7f8a29600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 0x7f8a2419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:31.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.398+0000 7f8a29600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a24109c20 0x7f8a241a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:31.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a29600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a241138e0 con 0x7f8a241092d0 2024-09-21T01:18:31.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a29600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24113760 con 0x7f8a24103990 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a29600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24113a60 con 0x7f8a24109c20 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a23400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 0x7f8a2419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a28c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a24109c20 0x7f8a241a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a23400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 0x7f8a2419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36248/0 (socket says 172.21.15.27:36248) 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a23400640 1 -- 172.21.15.27:0/4115704802 learned_addr learned my addr 172.21.15.27:0/4115704802 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a23e00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 0x7f8a2419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:31.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a28c00640 1 -- 172.21.15.27:0/4115704802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 msgr2=0x7f8a2419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a28c00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 0x7f8a2419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a28c00640 1 -- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 msgr2=0x7f8a2419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a28c00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 0x7f8a2419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a28c00640 1 -- 172.21.15.27:0/4115704802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a241a24a0 con 0x7f8a24109c20 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.399+0000 7f8a23400640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 0x7f8a2419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.400+0000 7f8a28c00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a24109c20 0x7f8a241a1d70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00ef00 tx=0x7f8a1c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.400+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c004b20 con 0x7f8a24109c20 2024-09-21T01:18:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.400+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c003e20 con 0x7f8a24109c20 2024-09-21T01:18:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.400+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241a2790 con 0x7f8a24109c20 2024-09-21T01:18:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.400+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00b570 con 0x7f8a24109c20 2024-09-21T01:18:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.400+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a241a8ef0 con 0x7f8a24109c20 2024-09-21T01:18:31.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.401+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a24103da0 con 0x7f8a24109c20 2024-09-21T01:18:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.404+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8a1c01d020 con 0x7f8a24109c20 2024-09-21T01:18:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.405+0000 7f8a20c00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8a080b65e0 0x7f8a080b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:31.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.405+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f8a1c0e7870 con 0x7f8a24109c20 2024-09-21T01:18:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.405+0000 7f8a23e00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8a080b65e0 0x7f8a080b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.406+0000 7f8a23e00640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8a080b65e0 0x7f8a080b8af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a1402a930 tx=0x7f8a140058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.407+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a1c024030 con 0x7f8a24109c20 2024-09-21T01:18:31.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6836/392681445,v1:172.21.15.27:6837/392681445] up:boot 2024-09-21T01:18:31.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: fsmap 4 up:standby 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:31.501 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:31.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: Deploying daemon mds.e on smithi089 2024-09-21T01:18:31.502 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:31 smithi089 ceph-mon[34486]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:31.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.643+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a241a9220 con 0x7f8a24109c20 2024-09-21T01:18:31.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6836/392681445,v1:172.21.15.27:6837/392681445] up:boot 2024-09-21T01:18:31.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: fsmap 4 up:standby 2024-09-21T01:18:31.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:18:31.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.644 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:31.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:31.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:31.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: Deploying daemon mds.e on smithi089 2024-09-21T01:18:31.645 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:31 smithi027 ceph-mon[30375]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.643+0000 7f8a20c00640 1 -- 172.21.15.27:0/4115704802 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3901 (secure 0 0 0) 0x7f8a1c004cc0 con 0x7f8a24109c20 2024-09-21T01:18:31.646 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 5 2024-09-21T01:18:31.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:31.646 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"btime":"2024-09-21T01:18:30:191239+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8a080b65e0 msgr2=0x7f8a080b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8a080b65e0 0x7f8a080b8af0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a1402a930 tx=0x7f8a140058d0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a24109c20 msgr2=0x7f8a241a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a24109c20 0x7f8a241a1d70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00ef00 tx=0x7f8a1c00c740 comp rx=0 tx=0).stop 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 shutdown_connections 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8a080b65e0 0x7f8a080b8af0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8a24109c20 0x7f8a241a1d70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a241092d0 0x7f8a2419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 --2- 172.21.15.27:0/4115704802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a24103990 0x7f8a2419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 >> 172.21.15.27:0/4115704802 conn(0x7f8a240ff3e0 msgr2=0x7f8a241089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.646+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 shutdown_connections 2024-09-21T01:18:31.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:31.647+0000 7f8a29600640 1 -- 172.21.15.27:0/4115704802 wait complete. 2024-09-21T01:18:31.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6836/392681445,v1:172.21.15.27:6837/392681445] up:boot 2024-09-21T01:18:31.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: fsmap 4 up:standby 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:31.650 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: Deploying daemon mds.e on smithi089 2024-09-21T01:18:31.651 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:31 smithi098 ceph-mon[34372]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:32.373 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:32 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4115704802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:32.421 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:32 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4115704802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:32.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:32 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4115704802' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:33.000 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:18:32 smithi089 systemd[1]: Started Ceph mds.e for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: Deploying daemon mds.f on smithi098 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6834/3850699909,v1:172.21.15.89:6835/3850699909] up:boot 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: fsmap 5 up:standby 2024-09-21T01:18:34.098 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:33 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: Deploying daemon mds.f on smithi098 2024-09-21T01:18:34.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6834/3850699909,v1:172.21.15.89:6835/3850699909] up:boot 2024-09-21T01:18:34.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: fsmap 5 up:standby 2024-09-21T01:18:34.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:33 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: Deploying daemon mds.f on smithi098 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6834/3850699909,v1:172.21.15.89:6835/3850699909] up:boot 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: fsmap 5 up:standby 2024-09-21T01:18:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:33 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:18:34.412 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:35.058 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:34 smithi098 systemd[1]: Starting Ceph mds.f for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:35.326 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 podman[63521]: 2024-09-21 01:18:34.96041546 +0000 UTC m=+0.015587535 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:35.326 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 podman[63521]: 2024-09-21 01:18:35.064173945 +0000 UTC m=+0.119346049 container create cd07f533729873916d67a1cddb058ee23838a9fe077a37dbbeec92fc3a3a7200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T01:18:35.326 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 podman[63521]: 2024-09-21 01:18:35.326047241 +0000 UTC m=+0.381219312 container init cd07f533729873916d67a1cddb058ee23838a9fe077a37dbbeec92fc3a3a7200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.467+0000 7f3102c00640 1 -- 172.21.15.27:0/321079842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc077940 msgr2=0x7f30fc077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.467+0000 7f3102c00640 1 --2- 172.21.15.27:0/321079842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc077940 0x7f30fc077d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009900 tx=0x7f30f002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 -- 172.21.15.27:0/321079842 shutdown_connections 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 --2- 172.21.15.27:0/321079842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc114570 0x7f30fc116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 --2- 172.21.15.27:0/321079842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc078290 0x7f30fc114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 --2- 172.21.15.27:0/321079842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc077940 0x7f30fc077d50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 -- 172.21.15.27:0/321079842 >> 172.21.15.27:0/321079842 conn(0x7f30fc101680 msgr2=0x7f30fc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 -- 172.21.15.27:0/321079842 shutdown_connections 2024-09-21T01:18:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.468+0000 7f3102c00640 1 -- 172.21.15.27:0/321079842 wait complete. 2024-09-21T01:18:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 Processor -- start 2024-09-21T01:18:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 -- start start 2024-09-21T01:18:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 0x7f30fc1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc078290 0x7f30fc1a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc114570 0x7f30fc1a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc119ef0 con 0x7f30fc078290 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc119d70 con 0x7f30fc114570 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.469+0000 7f3102c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc11a070 con 0x7f30fc077940 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 0x7f30fc1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 0x7f30fc1a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55114/0 (socket says 172.21.15.27:55114) 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 -- 172.21.15.27:0/3622190204 learned_addr learned my addr 172.21.15.27:0/3622190204 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3102200640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc114570 0x7f30fc1a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3100e00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc078290 0x7f30fc1a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 -- 172.21.15.27:0/3622190204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc114570 msgr2=0x7f30fc1a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc114570 0x7f30fc1a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 -- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc078290 msgr2=0x7f30fc1a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc078290 0x7f30fc1a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 -- 172.21.15.27:0/3622190204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc1a6870 con 0x7f30fc077940 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3100e00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc078290 0x7f30fc1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.470+0000 7f3101800640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 0x7f30fc1a0900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005940 tx=0x7f30f00040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:35.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.471+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f003f030 con 0x7f30fc077940 2024-09-21T01:18:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.471+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc1a6b00 con 0x7f30fc077940 2024-09-21T01:18:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.471+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f00044c0 con 0x7f30fc077940 2024-09-21T01:18:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.471+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0035070 con 0x7f30fc077940 2024-09-21T01:18:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.471+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1ad2c0 con 0x7f30fc077940 2024-09-21T01:18:35.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.472+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc077d50 con 0x7f30fc077940 2024-09-21T01:18:35.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.474+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f30f0042030 con 0x7f30fc077940 2024-09-21T01:18:35.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.476+0000 7f30fa000640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30e40b65e0 0x7f30e40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.476+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f30f0107630 con 0x7f30fc077940 2024-09-21T01:18:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.476+0000 7f3100e00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30e40b65e0 0x7f30e40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:35.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.477+0000 7f3100e00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30e40b65e0 0x7f30e40b8af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30fc079a30 tx=0x7f30e8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:35.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.477+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30f00ce1f0 con 0x7f30fc077940 2024-09-21T01:18:35.648 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 podman[63521]: 2024-09-21 01:18:35.329037623 +0000 UTC m=+0.384209686 container start cd07f533729873916d67a1cddb058ee23838a9fe077a37dbbeec92fc3a3a7200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T01:18:35.648 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mds[63542]: -- 172.21.15.98:0/1648828909 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563992bef860 con 0x563996d97a80 2024-09-21T01:18:35.648 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f[63538]: starting mds.f at 2024-09-21T01:18:35.648 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 bash[63521]: cd07f533729873916d67a1cddb058ee23838a9fe077a37dbbeec92fc3a3a7200 2024-09-21T01:18:35.648 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:18:35 smithi098 systemd[1]: Started Ceph mds.f for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:35.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.719+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30fc1a1a30 con 0x7f30fc077940 2024-09-21T01:18:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.720+0000 7f30fa000640 1 -- 172.21.15.27:0/3622190204 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4726 (secure 0 0 0) 0x7f30f00d25d0 con 0x7f30fc077940 2024-09-21T01:18:35.721 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:35.721 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":6,"btime":"2024-09-21T01:18:33:220995+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T01:18:35.721 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 6 2024-09-21T01:18:35.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30e40b65e0 msgr2=0x7f30e40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30e40b65e0 0x7f30e40b8af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30fc079a30 tx=0x7f30e8005f90 comp rx=0 tx=0).stop 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 msgr2=0x7f30fc1a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 0x7f30fc1a0900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005940 tx=0x7f30f00040c0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 shutdown_connections 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30e40b65e0 0x7f30e40b8af0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f30fc114570 0x7f30fc1a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc078290 0x7f30fc1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 --2- 172.21.15.27:0/3622190204 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30fc077940 0x7f30fc1a0900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 >> 172.21.15.27:0/3622190204 conn(0x7f30fc101680 msgr2=0x7f30fc116600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 shutdown_connections 2024-09-21T01:18:35.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:35.723+0000 7f3102c00640 1 -- 172.21.15.27:0/3622190204 wait complete. 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:36.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:36.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:35 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3622190204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:35 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3622190204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:35 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3622190204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:37.044 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:36 smithi098 ceph-mon[34372]: Deploying daemon mds.c on smithi098 2024-09-21T01:18:37.044 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:36 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6834/1764678372,v1:172.21.15.98:6835/1764678372] up:boot 2024-09-21T01:18:37.044 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:36 smithi098 ceph-mon[34372]: fsmap 6 up:standby 2024-09-21T01:18:37.044 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:36 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:18:37.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:36 smithi027 ceph-mon[30375]: Deploying daemon mds.c on smithi098 2024-09-21T01:18:37.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:36 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6834/1764678372,v1:172.21.15.98:6835/1764678372] up:boot 2024-09-21T01:18:37.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:36 smithi027 ceph-mon[30375]: fsmap 6 up:standby 2024-09-21T01:18:37.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:36 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:18:37.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:36 smithi089 ceph-mon[34486]: Deploying daemon mds.c on smithi098 2024-09-21T01:18:37.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:36 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6834/1764678372,v1:172.21.15.98:6835/1764678372] up:boot 2024-09-21T01:18:37.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:36 smithi089 ceph-mon[34486]: fsmap 6 up:standby 2024-09-21T01:18:37.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:36 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:18:37.650 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:37 smithi098 systemd[1]: Starting Ceph mds.c for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:38.016 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:37 smithi098 ceph-mon[34372]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:38.017 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:37 smithi098 podman[63787]: 2024-09-21 01:18:37.639806289 +0000 UTC m=+0.015962073 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:38.017 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:37 smithi098 podman[63787]: 2024-09-21 01:18:37.755388627 +0000 UTC m=+0.131544440 container create 124e199b88089c8c9cc4af22e3a54a09c784e5d56ef3ac120dbb026e1879fa1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-21T01:18:38.017 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:38 smithi098 podman[63787]: 2024-09-21 01:18:38.016852841 +0000 UTC m=+0.393008609 container init 124e199b88089c8c9cc4af22e3a54a09c784e5d56ef3ac120dbb026e1879fa1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T01:18:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:37 smithi027 ceph-mon[30375]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:38.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:37 smithi089 ceph-mon[34486]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:38.398 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:38 smithi098 podman[63787]: 2024-09-21 01:18:38.020131816 +0000 UTC m=+0.396287584 container start 124e199b88089c8c9cc4af22e3a54a09c784e5d56ef3ac120dbb026e1879fa1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:18:38.398 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:38 smithi098 ceph-mds[63806]: -- 172.21.15.98:0/1685614081 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556882915860 con 0x556886abda80 2024-09-21T01:18:38.398 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c[63802]: starting mds.c at 2024-09-21T01:18:38.398 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:38 smithi098 bash[63787]: 124e199b88089c8c9cc4af22e3a54a09c784e5d56ef3ac120dbb026e1879fa1c 2024-09-21T01:18:38.398 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:18:38 smithi098 systemd[1]: Started Ceph mds.c for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:38.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: Deploying daemon mds.h on smithi089 2024-09-21T01:18:39.417 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:39.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] up:boot 2024-09-21T01:18:39.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: fsmap 7 up:standby 2024-09-21T01:18:39.418 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:39 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: Deploying daemon mds.h on smithi089 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] up:boot 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: fsmap 7 up:standby 2024-09-21T01:18:39.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:39 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.600+0000 7f5413e00640 1 -- 172.21.15.27:0/1700421920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c078290 msgr2=0x7f540c114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.600+0000 7f5413e00640 1 --2- 172.21.15.27:0/1700421920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c078290 0x7f540c114200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009860 tx=0x7f53f802a340 comp rx=0 tx=0).stop 2024-09-21T01:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 -- 172.21.15.27:0/1700421920 shutdown_connections 2024-09-21T01:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 --2- 172.21.15.27:0/1700421920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f540c114740 0x7f540c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 --2- 172.21.15.27:0/1700421920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c078290 0x7f540c114200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 --2- 172.21.15.27:0/1700421920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 -- 172.21.15.27:0/1700421920 >> 172.21.15.27:0/1700421920 conn(0x7f540c101810 msgr2=0x7f540c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 -- 172.21.15.27:0/1700421920 shutdown_connections 2024-09-21T01:18:39.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.601+0000 7f5413e00640 1 -- 172.21.15.27:0/1700421920 wait complete. 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 Processor -- start 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 -- start start 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f540c078290 0x7f540c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c114740 0x7f540c1a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f540c11a090 con 0x7f540c077940 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f540c119f10 con 0x7f540c114740 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.602+0000 7f5413e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f540c11a210 con 0x7f540c078290 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c1a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35466/0 (socket says 172.21.15.27:35466) 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 -- 172.21.15.27:0/3092075302 learned_addr learned my addr 172.21.15.27:0/3092075302 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5410a00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f540c078290 0x7f540c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c114740 0x7f540c1a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 -- 172.21.15.27:0/3092075302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f540c078290 msgr2=0x7f540c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f540c078290 0x7f540c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 -- 172.21.15.27:0/3092075302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c114740 msgr2=0x7f540c1a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c114740 0x7f540c1a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.603+0000 7f5411400640 1 -- 172.21.15.27:0/3092075302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1a6a10 con 0x7f540c077940 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.604+0000 7f5411400640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c1a0aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f540000e9c0 tx=0x7f540000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.604+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540000cde0 con 0x7f540c077940 2024-09-21T01:18:39.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.604+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54000035f0 con 0x7f540c077940 2024-09-21T01:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.604+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5400010690 con 0x7f540c077940 2024-09-21T01:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.604+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1a6d00 con 0x7f540c077940 2024-09-21T01:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.604+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1ad460 con 0x7f540c077940 2024-09-21T01:18:39.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.605+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c077d50 con 0x7f540c077940 2024-09-21T01:18:39.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.608+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f5400004000 con 0x7f540c077940 2024-09-21T01:18:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.609+0000 7f540a000640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f53f40b65e0 0x7f53f40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.609+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f54000e7390 con 0x7f540c077940 2024-09-21T01:18:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.609+0000 7f5410a00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f53f40b65e0 0x7f53f40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:39.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.610+0000 7f5410a00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f53f40b65e0 0x7f53f40b8af0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53f8005940 tx=0x7f53f80058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:39.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.611+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54000adfe0 con 0x7f540c077940 2024-09-21T01:18:39.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: Deploying daemon mds.h on smithi089 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] up:boot 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: fsmap 7 up:standby 2024-09-21T01:18:39.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:39 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:18:39.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.848+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f540c1a1bd0 con 0x7f540c077940 2024-09-21T01:18:39.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.848+0000 7f540a000640 1 -- 172.21.15.27:0/3092075302 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6376 (secure 0 0 0) 0x7f54000b23c0 con 0x7f540c077940 2024-09-21T01:18:39.849 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:39.849 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"btime":"2024-09-21T01:18:38:864365+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T01:18:39.849 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2024-09-21T01:18:39.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f53f40b65e0 msgr2=0x7f53f40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:39.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f53f40b65e0 0x7f53f40b8af0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53f8005940 tx=0x7f53f80058d0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 msgr2=0x7f540c1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c1a0aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f540000e9c0 tx=0x7f540000ee80 comp rx=0 tx=0).stop 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 shutdown_connections 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f53f40b65e0 0x7f53f40b8af0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f540c114740 0x7f540c1a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f540c078290 0x7f540c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 --2- 172.21.15.27:0/3092075302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f540c077940 0x7f540c1a0aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 >> 172.21.15.27:0/3092075302 conn(0x7f540c101810 msgr2=0x7f540c1167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 shutdown_connections 2024-09-21T01:18:39.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:39.851+0000 7f5413e00640 1 -- 172.21.15.27:0/3092075302 wait complete. 2024-09-21T01:18:40.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:40 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3092075302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:40 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3092075302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:40 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3092075302' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:41.000 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:18:40 smithi089 systemd[1]: Started Ceph mds.h for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: Deploying daemon mds.i on smithi098 2024-09-21T01:18:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] up:boot 2024-09-21T01:18:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: fsmap 8 up:standby 2024-09-21T01:18:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:18:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:42.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:42.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:42.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: Deploying daemon mds.i on smithi098 2024-09-21T01:18:42.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] up:boot 2024-09-21T01:18:42.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: fsmap 8 up:standby 2024-09-21T01:18:42.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: Deploying daemon mds.i on smithi098 2024-09-21T01:18:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] up:boot 2024-09-21T01:18:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: fsmap 8 up:standby 2024-09-21T01:18:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:18:42.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:42.898 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:42 smithi098 systemd[1]: Starting Ceph mds.i for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:43.292 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:42 smithi098 podman[64059]: 2024-09-21 01:18:42.891844586 +0000 UTC m=+0.015723953 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:43.292 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 podman[64059]: 2024-09-21 01:18:43.039623822 +0000 UTC m=+0.163503230 container create 5edfd74fe830e5c65f634d8b906622e5cf9f4ac26ef8a71beb5afce946924c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:18:43.292 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 podman[64059]: 2024-09-21 01:18:43.291657067 +0000 UTC m=+0.415536432 container init 5edfd74fe830e5c65f634d8b906622e5cf9f4ac26ef8a71beb5afce946924c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:18:43.648 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 podman[64059]: 2024-09-21 01:18:43.294972603 +0000 UTC m=+0.418851979 container start 5edfd74fe830e5c65f634d8b906622e5cf9f4ac26ef8a71beb5afce946924c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T01:18:43.648 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mds[64079]: -- 172.21.15.98:0/4074585058 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598d6969860 con 0x5598dab11600 2024-09-21T01:18:43.648 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i[64075]: starting mds.i at 2024-09-21T01:18:43.648 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 bash[64059]: 5edfd74fe830e5c65f634d8b906622e5cf9f4ac26ef8a71beb5afce946924c37 2024-09-21T01:18:43.648 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:18:43 smithi098 systemd[1]: Started Ceph mds.i for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:43.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 -- 172.21.15.27:0/3357863719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e410b6d0 msgr2=0x7f65e410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 --2- 172.21.15.27:0/3357863719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e410b6d0 0x7f65e410bae0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009900 tx=0x7f65d002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 -- 172.21.15.27:0/3357863719 shutdown_connections 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 --2- 172.21.15.27:0/3357863719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 0x7f65e4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 --2- 172.21.15.27:0/3357863719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e410b6d0 0x7f65e410bae0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 --2- 172.21.15.27:0/3357863719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e4105d90 0x7f65e41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 -- 172.21.15.27:0/3357863719 >> 172.21.15.27:0/3357863719 conn(0x7f65e4101860 msgr2=0x7f65e4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.769+0000 7f65e9a00640 1 -- 172.21.15.27:0/3357863719 shutdown_connections 2024-09-21T01:18:43.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.770+0000 7f65e9a00640 1 -- 172.21.15.27:0/3357863719 wait complete. 2024-09-21T01:18:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 Processor -- start 2024-09-21T01:18:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 -- start start 2024-09-21T01:18:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 0x7f65e419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e410b6d0 0x7f65e419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 0x7f65e41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:43.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4115d30 con 0x7f65e410b6d0 2024-09-21T01:18:43.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4115bb0 con 0x7f65e410c020 2024-09-21T01:18:43.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.771+0000 7f65e9a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4115eb0 con 0x7f65e4105d90 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 0x7f65e419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e410b6d0 0x7f65e419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 0x7f65e419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:50876/0 (socket says 172.21.15.27:50876) 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e410b6d0 0x7f65e419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35472/0 (socket says 172.21.15.27:35472) 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 -- 172.21.15.27:0/1655273686 learned_addr learned my addr 172.21.15.27:0/1655273686 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e3400640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 0x7f65e41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 -- 172.21.15.27:0/1655273686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 msgr2=0x7f65e41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 0x7f65e41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 -- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e410b6d0 msgr2=0x7f65e419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e410b6d0 0x7f65e419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:43.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e2a00640 1 -- 172.21.15.27:0/1655273686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e41a2630 con 0x7f65e4105d90 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.772+0000 7f65e3400640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 0x7f65e41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.773+0000 7f65e2a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 0x7f65e419c6c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65d400e9f0 tx=0x7f65d400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.773+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d400cd80 con 0x7f65e4105d90 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.773+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d40108e0 con 0x7f65e4105d90 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.773+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e41a2920 con 0x7f65e4105d90 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.773+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4018980 con 0x7f65e4105d90 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.773+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41a9080 con 0x7f65e4105d90 2024-09-21T01:18:43.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.774+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e41061a0 con 0x7f65e4105d90 2024-09-21T01:18:43.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.776+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f65d4010430 con 0x7f65e4105d90 2024-09-21T01:18:43.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.778+0000 7f65db400640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f65c80b65e0 0x7f65c80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:43.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.778+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f65d40e73a0 con 0x7f65e4105d90 2024-09-21T01:18:43.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.778+0000 7f65e2000640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f65c80b65e0 0x7f65c80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:43.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.779+0000 7f65e2000640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f65c80b65e0 0x7f65c80b8af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65d002a9e0 tx=0x7f65d0005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:43.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:43.781+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f65d40adf60 con 0x7f65e4105d90 2024-09-21T01:18:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:43.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:43.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:43.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:43 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:44.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.042+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65e419d7f0 con 0x7f65e4105d90 2024-09-21T01:18:44.044 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.043+0000 7f65db400640 1 -- 172.21.15.27:0/1655273686 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8028 (secure 0 0 0) 0x7f65d40b2340 con 0x7f65e4105d90 2024-09-21T01:18:44.044 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:44.044 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"btime":"2024-09-21T01:18:43:794512+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24734,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T01:18:44.045 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 10 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f65c80b65e0 msgr2=0x7f65c80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f65c80b65e0 0x7f65c80b8af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65d002a9e0 tx=0x7f65d0005a20 comp rx=0 tx=0).stop 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 msgr2=0x7f65e419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 0x7f65e419c6c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65d400e9f0 tx=0x7f65d400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 shutdown_connections 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f65c80b65e0 0x7f65c80b8af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f65e410c020 0x7f65e41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65e410b6d0 0x7f65e419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 --2- 172.21.15.27:0/1655273686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65e4105d90 0x7f65e419c6c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 >> 172.21.15.27:0/1655273686 conn(0x7f65e4101860 msgr2=0x7f65e4108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:44.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 shutdown_connections 2024-09-21T01:18:44.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:44.046+0000 7f65e9a00640 1 -- 172.21.15.27:0/1655273686 wait complete. 2024-09-21T01:18:44.130 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:44.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:44.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:44.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:44.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:44.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:44.131 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:43 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:43 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:45.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:44 smithi089 ceph-mon[34486]: Deploying daemon mds.g on smithi027 2024-09-21T01:18:45.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:44 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] up:boot 2024-09-21T01:18:45.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:44 smithi089 ceph-mon[34486]: fsmap 9 up:standby 2024-09-21T01:18:45.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:44 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:18:45.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:44 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1655273686' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:45.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:44 smithi027 ceph-mon[30375]: Deploying daemon mds.g on smithi027 2024-09-21T01:18:45.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:44 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] up:boot 2024-09-21T01:18:45.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:44 smithi027 ceph-mon[30375]: fsmap 9 up:standby 2024-09-21T01:18:45.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:44 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:18:45.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:44 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1655273686' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:45.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:44 smithi098 ceph-mon[34372]: Deploying daemon mds.g on smithi027 2024-09-21T01:18:45.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:44 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] up:boot 2024-09-21T01:18:45.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:44 smithi098 ceph-mon[34372]: fsmap 9 up:standby 2024-09-21T01:18:45.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:44 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:18:45.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:44 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1655273686' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:45 smithi098 ceph-mon[34372]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:45 smithi027 ceph-mon[30375]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:46.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:45 smithi089 ceph-mon[34486]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:46.816 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:47.291 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 podman[73062]: 2024-09-21 01:18:47.037227056 +0000 UTC m=+0.157601274 container create 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T01:18:47.291 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 podman[73062]: 2024-09-21 01:18:47.290809849 +0000 UTC m=+0.411184099 container init 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:18:47.507 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:47.596 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 podman[73062]: 2024-09-21 01:18:47.297551864 +0000 UTC m=+0.417926092 container start 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:18:47.596 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mds[73104]: -- 172.21.15.27:0/3982848482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ce26cf860 con 0x561ce6877a80 2024-09-21T01:18:47.596 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 bash[73062]: 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe 2024-09-21T01:18:47.596 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g[73090]: starting mds.g at 2024-09-21T01:18:47.597 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:18:47 smithi027 systemd[1]: Started Ceph mds.g for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:47.858 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:47 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:47 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:48.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 -- 172.21.15.27:0/3211263844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 msgr2=0x7fa3f4103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 --2- 172.21.15.27:0/3211263844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f4103da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009900 tx=0x7fa3e802a410 comp rx=0 tx=0).stop 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 -- 172.21.15.27:0/3211263844 shutdown_connections 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 --2- 172.21.15.27:0/3211263844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 --2- 172.21.15.27:0/3211263844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3f41092d0 0x7fa3f41096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 --2- 172.21.15.27:0/3211263844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f4103da0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 -- 172.21.15.27:0/3211263844 >> 172.21.15.27:0/3211263844 conn(0x7fa3f40ff440 msgr2=0x7fa3f4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.164+0000 7fa3fb000640 1 -- 172.21.15.27:0/3211263844 shutdown_connections 2024-09-21T01:18:48.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.165+0000 7fa3fb000640 1 -- 172.21.15.27:0/3211263844 wait complete. 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 Processor -- start 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 -- start start 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3f41092d0 0x7fa3f419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f41a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113930 con 0x7fa3f4103990 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41137b0 con 0x7fa3f4109c20 2024-09-21T01:18:48.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.166+0000 7fa3fb000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113ab0 con 0x7fa3f41092d0 2024-09-21T01:18:48.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f41a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3f9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52896/0 (socket says 172.21.15.27:52896) 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3f9200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3f41092d0 0x7fa3f419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f41a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40206/0 (socket says 172.21.15.27:40206) 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3f9c00640 1 -- 172.21.15.27:0/2121005777 learned_addr learned my addr 172.21.15.27:0/2121005777 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 -- 172.21.15.27:0/2121005777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3f41092d0 msgr2=0x7fa3f419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3f41092d0 0x7fa3f419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 -- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 msgr2=0x7fa3f419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3fa600640 1 -- 172.21.15.27:0/2121005777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41a24b0 con 0x7fa3f4109c20 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.167+0000 7fa3f9c00640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.168+0000 7fa3fa600640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f41a1d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009fd0 tx=0x7fa3ec00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:48.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.168+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00ce20 con 0x7fa3f4109c20 2024-09-21T01:18:48.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.168+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f41a27a0 con 0x7fa3f4109c20 2024-09-21T01:18:48.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.168+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ec005020 con 0x7fa3f4109c20 2024-09-21T01:18:48.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.168+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec010690 con 0x7fa3f4109c20 2024-09-21T01:18:48.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.168+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a8f00 con 0x7fa3f4109c20 2024-09-21T01:18:48.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.169+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f4103da0 con 0x7fa3f4109c20 2024-09-21T01:18:48.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.172+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fa3ec0054d0 con 0x7fa3f4109c20 2024-09-21T01:18:48.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.173+0000 7fa3f2000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:48.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.173+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa3ec0e7680 con 0x7fa3f4109c20 2024-09-21T01:18:48.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.173+0000 7fa3f9c00640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:48.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.174+0000 7fa3f9c00640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80096c0 tx=0x7fa3e8005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:48.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.175+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3ec0ae240 con 0x7fa3f4109c20 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:47 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:48.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.415+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f419d670 con 0x7fa3f4109c20 2024-09-21T01:18:48.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.416+0000 7fa3f2000640 1 -- 172.21.15.27:0/2121005777 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8854 (secure 0 0 0) 0x7fa3f419d670 con 0x7fa3f4109c20 2024-09-21T01:18:48.418 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:48.418 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":11,"btime":"2024-09-21T01:18:47:809941+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24734,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24752,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-21T01:18:48.419 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa3dc0b6560 msgr2=0x7fa3dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80096c0 tx=0x7fa3e8005a20 comp rx=0 tx=0).stop 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 msgr2=0x7fa3f41a1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f41a1d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009fd0 tx=0x7fa3ec00ef10 comp rx=0 tx=0).stop 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 shutdown_connections 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa3f4109c20 0x7fa3f41a1d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa3f41092d0 0x7fa3f419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 --2- 172.21.15.27:0/2121005777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4103990 0x7fa3f419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 >> 172.21.15.27:0/2121005777 conn(0x7fa3f40ff440 msgr2=0x7fa3f4108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 shutdown_connections 2024-09-21T01:18:48.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:48.419+0000 7fa3fb000640 1 -- 172.21.15.27:0/2121005777 wait complete. 2024-09-21T01:18:49.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:48 smithi098 ceph-mon[34372]: Deploying daemon mds.a on smithi027 2024-09-21T01:18:49.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:48 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] up:boot 2024-09-21T01:18:49.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:48 smithi098 ceph-mon[34372]: fsmap 10 up:standby 2024-09-21T01:18:49.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:48 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:18:49.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:48 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2121005777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:49.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:48 smithi027 ceph-mon[30375]: Deploying daemon mds.a on smithi027 2024-09-21T01:18:49.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:48 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] up:boot 2024-09-21T01:18:49.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:48 smithi027 ceph-mon[30375]: fsmap 10 up:standby 2024-09-21T01:18:49.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:48 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:18:49.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:48 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2121005777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:49.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:48 smithi089 ceph-mon[34486]: Deploying daemon mds.a on smithi027 2024-09-21T01:18:49.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:48 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] up:boot 2024-09-21T01:18:49.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:48 smithi089 ceph-mon[34486]: fsmap 10 up:standby 2024-09-21T01:18:49.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:48 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:18:49.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:48 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2121005777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:49.944 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:49 smithi027 ceph-mon[30375]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:49 smithi098 ceph-mon[34372]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:49 smithi089 ceph-mon[34486]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:51.296 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:50 smithi027 systemd[1]: Starting Ceph mds.a for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:51.592 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 podman[73422]: 2024-09-21 01:18:51.199666618 +0000 UTC m=+0.027959385 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:51.593 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 podman[73422]: 2024-09-21 01:18:51.329976501 +0000 UTC m=+0.158269229 container create a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:18:51.593 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 podman[73422]: 2024-09-21 01:18:51.59241276 +0000 UTC m=+0.420705524 container init a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T01:18:51.730 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:51.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:51.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:51.932 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 podman[73422]: 2024-09-21 01:18:51.598644397 +0000 UTC m=+0.426937125 container start a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T01:18:51.932 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 bash[73422]: a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 2024-09-21T01:18:51.932 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 systemd[1]: Started Ceph mds.a for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:51.932 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-mds[73441]: -- 172.21.15.27:0/1271748183 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597cef31860 con 0x5597d30d9600 2024-09-21T01:18:51.932 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:18:51 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a[73437]: starting mds.a at 2024-09-21T01:18:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:51 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:52.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:52.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:52.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:52.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:52.179 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T01:18:52.180 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T01:18:52.180 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:51 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 -- 172.21.15.27:0/230379306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 msgr2=0x7f6edc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 --2- 172.21.15.27:0/230379306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 0x7f6edc106190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009900 tx=0x7f6ecc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 -- 172.21.15.27:0/230379306 shutdown_connections 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 --2- 172.21.15.27:0/230379306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10c010 0x7f6edc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 --2- 172.21.15.27:0/230379306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6edc10b6c0 0x7f6edc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 --2- 172.21.15.27:0/230379306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 0x7f6edc106190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 -- 172.21.15.27:0/230379306 >> 172.21.15.27:0/230379306 conn(0x7f6edc101810 msgr2=0x7f6edc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 -- 172.21.15.27:0/230379306 shutdown_connections 2024-09-21T01:18:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.787+0000 7f6ee3000640 1 -- 172.21.15.27:0/230379306 wait complete. 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.788+0000 7f6ee3000640 1 Processor -- start 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.788+0000 7f6ee3000640 1 -- start start 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee3000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 0x7f6edc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee3000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 0x7f6edc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee3000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6edc10c010 0x7f6edc1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee3000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc115d20 con 0x7f6edc105d80 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee3000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc115ba0 con 0x7f6edc10b6c0 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee3000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc115ea0 con 0x7f6edc10c010 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6edb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 0x7f6edc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6edbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 0x7f6edc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6edb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 0x7f6edc19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40232/0 (socket says 172.21.15.27:40232) 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6edb400640 1 -- 172.21.15.27:0/2488340084 learned_addr learned my addr 172.21.15.27:0/2488340084 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.789+0000 7f6ee1000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6edc10c010 0x7f6edc1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6edb400640 1 -- 172.21.15.27:0/2488340084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6edc10c010 msgr2=0x7f6edc1a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6edb400640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6edc10c010 0x7f6edc1a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6edb400640 1 -- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 msgr2=0x7f6edc19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6edb400640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 0x7f6edc19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6edb400640 1 -- 172.21.15.27:0/2488340084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc1a2660 con 0x7f6edc10b6c0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6edb400640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 0x7f6edc19cc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800e9c0 tx=0x7f6ec800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800ccc0 con 0x7f6edc10b6c0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec80054d0 con 0x7f6edc10b6c0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8010430 con 0x7f6edc10b6c0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc1a2950 con 0x7f6edc10b6c0 2024-09-21T01:18:52.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.790+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc1a90b0 con 0x7f6edc10b6c0 2024-09-21T01:18:52.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.791+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc106190 con 0x7f6edc10b6c0 2024-09-21T01:18:52.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.794+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f6ec8005000 con 0x7f6edc10b6c0 2024-09-21T01:18:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.795+0000 7f6ed8c00640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ec00b65e0 0x7f6ec00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.796+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6ec80e75a0 con 0x7f6edc10b6c0 2024-09-21T01:18:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.796+0000 7f6edbe00640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ec00b65e0 0x7f6ec00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.796+0000 7f6edbe00640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ec00b65e0 0x7f6ec00b8af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005ab0 tx=0x7f6ecc005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:52.797+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ec8056710 con 0x7f6edc10b6c0 2024-09-21T01:18:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:52 smithi027 ceph-mon[30375]: Deploying daemon mds.b on smithi089 2024-09-21T01:18:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:52 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6840/2646952373,v1:172.21.15.27:6841/2646952373] up:boot 2024-09-21T01:18:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:52 smithi027 ceph-mon[30375]: fsmap 11 up:standby 2024-09-21T01:18:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:52 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T01:18:53.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.035+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6edc19d820 con 0x7f6edc10b6c0 2024-09-21T01:18:53.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.036+0000 7f6ed8c00640 1 -- 172.21.15.27:0/2488340084 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9680 (secure 0 0 0) 0x7f6edc19d820 con 0x7f6edc10b6c0 2024-09-21T01:18:53.037 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:53.037 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":12,"btime":"2024-09-21T01:18:51:827923+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24734,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24752,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-21T01:18:53.037 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 12 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ec00b65e0 msgr2=0x7f6ec00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ec00b65e0 0x7f6ec00b8af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005ab0 tx=0x7f6ecc005a20 comp rx=0 tx=0).stop 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 msgr2=0x7f6edc19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 0x7f6edc19cc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800e9c0 tx=0x7f6ec800ee80 comp rx=0 tx=0).stop 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 shutdown_connections 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ec00b65e0 0x7f6ec00b8af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6edc10c010 0x7f6edc1a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6edc10b6c0 0x7f6edc19cc90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 --2- 172.21.15.27:0/2488340084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6edc105d80 0x7f6edc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 >> 172.21.15.27:0/2488340084 conn(0x7f6edc101810 msgr2=0x7f6edc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 shutdown_connections 2024-09-21T01:18:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:53.038+0000 7f6ee3000640 1 -- 172.21.15.27:0/2488340084 wait complete. 2024-09-21T01:18:53.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:52 smithi098 ceph-mon[34372]: Deploying daemon mds.b on smithi089 2024-09-21T01:18:53.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:52 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6840/2646952373,v1:172.21.15.27:6841/2646952373] up:boot 2024-09-21T01:18:53.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:52 smithi098 ceph-mon[34372]: fsmap 11 up:standby 2024-09-21T01:18:53.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:52 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T01:18:53.226 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:52 smithi089 ceph-mon[34486]: Deploying daemon mds.b on smithi089 2024-09-21T01:18:53.227 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:52 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6840/2646952373,v1:172.21.15.27:6841/2646952373] up:boot 2024-09-21T01:18:53.227 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:52 smithi089 ceph-mon[34486]: fsmap 11 up:standby 2024-09-21T01:18:53.227 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:52 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T01:18:53.844 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:53 smithi089 systemd[1]: Starting Ceph mds.b for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:18:54.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:53 smithi098 ceph-mon[34372]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:54.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:53 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2488340084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:53 smithi027 ceph-mon[30375]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:53 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2488340084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:54.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:53 smithi089 ceph-mon[34486]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:54.183 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:53 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2488340084' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:54.184 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:53 smithi089 podman[63385]: 2024-09-21 01:18:53.807824008 +0000 UTC m=+0.015750310 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:18:54.184 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:53 smithi089 podman[63385]: 2024-09-21 01:18:53.93747859 +0000 UTC m=+0.145404886 container create 2867b76e2e4f7814892dd50edf84c90166bce9f24b95ba8b1ef00df7c8e13bc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T01:18:54.184 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:54 smithi089 podman[63385]: 2024-09-21 01:18:54.184090603 +0000 UTC m=+0.392016896 container init 2867b76e2e4f7814892dd50edf84c90166bce9f24b95ba8b1ef00df7c8e13bc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T01:18:54.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:54 smithi089 podman[63385]: 2024-09-21 01:18:54.186958626 +0000 UTC m=+0.394884920 container start 2867b76e2e4f7814892dd50edf84c90166bce9f24b95ba8b1ef00df7c8e13bc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T01:18:54.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:54 smithi089 ceph-mds[63405]: -- 172.21.15.89:0/955696039 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa3e8b7860 con 0x55aa42a5fa80 2024-09-21T01:18:54.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:54 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: starting mds.b at 2024-09-21T01:18:54.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:54 smithi089 bash[63385]: 2867b76e2e4f7814892dd50edf84c90166bce9f24b95ba8b1ef00df7c8e13bc4 2024-09-21T01:18:54.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:18:54 smithi089 systemd[1]: Started Ceph mds.b for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:18:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] up:boot 2024-09-21T01:18:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: fsmap 12 up:standby 2024-09-21T01:18:55.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:55 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:18:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:55.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:55.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] up:boot 2024-09-21T01:18:55.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: fsmap 12 up:standby 2024-09-21T01:18:55.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:55 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:55.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:55.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] up:boot 2024-09-21T01:18:55.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: fsmap 12 up:standby 2024-09-21T01:18:55.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:55 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:18:55.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:18:56.395 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:56.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:56.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:56.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:56.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:56.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:56 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:56 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 -- 172.21.15.27:0/2919713025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd80757b0 msgr2=0x7fcfd8075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 --2- 172.21.15.27:0/2919713025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd80757b0 0x7fcfd8075bc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009900 tx=0x7fcfcc02a410 comp rx=0 tx=0).stop 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 -- 172.21.15.27:0/2919713025 shutdown_connections 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 --2- 172.21.15.27:0/2919713025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd810ff10 0x7fcfd8112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 --2- 172.21.15.27:0/2919713025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 0x7fcfd810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 --2- 172.21.15.27:0/2919713025 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd80757b0 0x7fcfd8075bc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 -- 172.21.15.27:0/2919713025 >> 172.21.15.27:0/2919713025 conn(0x7fcfd80ff440 msgr2=0x7fcfd8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.056+0000 7fcfde200640 1 -- 172.21.15.27:0/2919713025 shutdown_connections 2024-09-21T01:18:57.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.057+0000 7fcfde200640 1 -- 172.21.15.27:0/2919713025 wait complete. 2024-09-21T01:18:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.058+0000 7fcfde200640 1 Processor -- start 2024-09-21T01:18:57.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.058+0000 7fcfde200640 1 -- start start 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfde200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 0x7fcfd819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfde200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 0x7fcfd819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfde200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd810ff10 0x7fcfd81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfde200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8115b10 con 0x7fcfd80757b0 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfde200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8115990 con 0x7fcfd810ff10 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfde200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8115c90 con 0x7fcfd8076100 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfdce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 0x7fcfd819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfdce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 0x7fcfd819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55700/0 (socket says 172.21.15.27:55700) 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfdce00640 1 -- 172.21.15.27:0/2992917031 learned_addr learned my addr 172.21.15.27:0/2992917031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfd7e00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 0x7fcfd819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:57.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.059+0000 7fcfdd800640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd810ff10 0x7fcfd81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd7e00640 1 -- 172.21.15.27:0/2992917031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd810ff10 msgr2=0x7fcfd81a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:57.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd7e00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd810ff10 0x7fcfd81a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd7e00640 1 -- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 msgr2=0x7fcfd819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd7e00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 0x7fcfd819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd7e00640 1 -- 172.21.15.27:0/2992917031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd81a2410 con 0x7fcfd8076100 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfdd800640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd810ff10 0x7fcfd81a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfdce00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 0x7fcfd819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd7e00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 0x7fcfd819ca40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400e9c0 tx=0x7fcfc400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400ccc0 con 0x7fcfd8076100 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc40035f0 con 0x7fcfd8076100 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400ce60 con 0x7fcfd8076100 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.060+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd81a2700 con 0x7fcfd8076100 2024-09-21T01:18:57.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.061+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd81a8e60 con 0x7fcfd8076100 2024-09-21T01:18:57.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.062+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd8075bc0 con 0x7fcfd8076100 2024-09-21T01:18:57.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.065+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fcfc4004000 con 0x7fcfd8076100 2024-09-21T01:18:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.067+0000 7fcfd5600640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcfc00b65e0 0x7fcfc00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:57.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.067+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fcfc40e76b0 con 0x7fcfd8076100 2024-09-21T01:18:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.067+0000 7fcfdce00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcfc00b65e0 0x7fcfc00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.068+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcfc40ae270 con 0x7fcfd8076100 2024-09-21T01:18:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.068+0000 7fcfdce00640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcfc00b65e0 0x7fcfc00b8af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0096c0 tx=0x7fcfcc005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:56 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.320+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfd819d5d0 con 0x7fcfd8076100 2024-09-21T01:18:57.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.321+0000 7fcfd5600640 1 -- 172.21.15.27:0/2992917031 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10506 (secure 0 0 0) 0x7fcfd819d5d0 con 0x7fcfd8076100 2024-09-21T01:18:57.321 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:18:57.322 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"btime":"2024-09-21T01:18:54:843234+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24734,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24752,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24776,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T01:18:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcfc00b65e0 msgr2=0x7fcfc00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcfc00b65e0 0x7fcfc00b8af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0096c0 tx=0x7fcfcc005a20 comp rx=0 tx=0).stop 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 msgr2=0x7fcfd819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 0x7fcfd819ca40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400e9c0 tx=0x7fcfc400ee80 comp rx=0 tx=0).stop 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 shutdown_connections 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcfc00b65e0 0x7fcfc00b8af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcfd810ff10 0x7fcfd81a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfd8076100 0x7fcfd819ca40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 --2- 172.21.15.27:0/2992917031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfd80757b0 0x7fcfd819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:57.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 >> 172.21.15.27:0/2992917031 conn(0x7fcfd80ff440 msgr2=0x7fcfd8109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:57.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 shutdown_connections 2024-09-21T01:18:57.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:57.323+0000 7fcfde200640 1 -- 172.21.15.27:0/2992917031 wait complete. 2024-09-21T01:18:58.160 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:58 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2992917031' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:58 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2992917031' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:18:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:58 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2992917031' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:18:59.310 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:18:59.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:59 smithi027 ceph-mon[30375]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:59.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.773 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:18:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:59 smithi098 ceph-mon[34372]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:18:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:18:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:18:59.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 -- 172.21.15.27:0/3991629050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c10c010 msgr2=0x7f441c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 --2- 172.21.15.27:0/3991629050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c10c010 0x7f441c112840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4414009ca0 tx=0x7f441402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 -- 172.21.15.27:0/3991629050 shutdown_connections 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 --2- 172.21.15.27:0/3991629050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c10c010 0x7f441c112840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 --2- 172.21.15.27:0/3991629050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 0x7f441c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 --2- 172.21.15.27:0/3991629050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f441c105d80 0x7f441c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 -- 172.21.15.27:0/3991629050 >> 172.21.15.27:0/3991629050 conn(0x7f441c101810 msgr2=0x7f441c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 -- 172.21.15.27:0/3991629050 shutdown_connections 2024-09-21T01:18:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.957+0000 7f4424600640 1 -- 172.21.15.27:0/3991629050 wait complete. 2024-09-21T01:18:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.958+0000 7f4424600640 1 Processor -- start 2024-09-21T01:18:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 -- start start 2024-09-21T01:18:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 0x7f441c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 0x7f441c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f441c10c010 0x7f441c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f441c115cd0 con 0x7f441c105d80 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f441c115b50 con 0x7f441c10b6c0 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4424600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f441c115e50 con 0x7f441c10c010 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4421c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 0x7f441c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4421200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 0x7f441c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4421c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 0x7f441c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55718/0 (socket says 172.21.15.27:55718) 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4421c00640 1 -- 172.21.15.27:0/891182334 learned_addr learned my addr 172.21.15.27:0/891182334 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.959+0000 7f4422600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f441c10c010 0x7f441c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421200640 1 -- 172.21.15.27:0/891182334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f441c10c010 msgr2=0x7f441c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421200640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f441c10c010 0x7f441c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421200640 1 -- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 msgr2=0x7f441c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:18:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421200640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 0x7f441c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421200640 1 -- 172.21.15.27:0/891182334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c1a25d0 con 0x7f441c10b6c0 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421c00640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 0x7f441c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.960+0000 7f4421200640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 0x7f441c19cc00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f441c06caa0 tx=0x7f440800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.961+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440800cc40 con 0x7f441c10b6c0 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.961+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c1a28c0 con 0x7f441c10b6c0 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.961+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440800cde0 con 0x7f441c10b6c0 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.961+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408018760 con 0x7f441c10b6c0 2024-09-21T01:18:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.961+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a9020 con 0x7f441c10b6c0 2024-09-21T01:18:59.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.962+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441c106190 con 0x7f441c10b6c0 2024-09-21T01:18:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.964+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f4408010a90 con 0x7f441c10b6c0 2024-09-21T01:18:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.966+0000 7f441a000640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44040b65e0 0x7f44040b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:18:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.966+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f44080e75e0 con 0x7f441c10b6c0 2024-09-21T01:18:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.966+0000 7f4421c00640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44040b65e0 0x7f44040b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:18:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.967+0000 7f4421c00640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44040b65e0 0x7f44040b8af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ce0 tx=0x7f4410005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:18:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:18:59.968+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44080ae1a0 con 0x7f441c10b6c0 2024-09-21T01:19:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:59 smithi089 ceph-mon[34486]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:18:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:00.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.176+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f441c19d790 con 0x7f441c10b6c0 2024-09-21T01:19:00.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.177+0000 7f441a000640 1 -- 172.21.15.27:0/891182334 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f44080d1120 con 0x7f441c10b6c0 2024-09-21T01:19:00.178 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:00.178 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-09-21T01:19:00.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:19:00.178 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-09-21T01:19:00.178 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-09-21T01:19:00.179 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:19:00.179 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44040b65e0 msgr2=0x7f44040b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44040b65e0 0x7f44040b8af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ce0 tx=0x7f4410005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 msgr2=0x7f441c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 0x7f441c19cc00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f441c06caa0 tx=0x7f440800ee50 comp rx=0 tx=0).stop 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 shutdown_connections 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44040b65e0 0x7f44040b8af0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f441c10c010 0x7f441c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f441c10b6c0 0x7f441c19cc00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 --2- 172.21.15.27:0/891182334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f441c105d80 0x7f441c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.180+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 >> 172.21.15.27:0/891182334 conn(0x7f441c101810 msgr2=0x7f441c108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.181+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 shutdown_connections 2024-09-21T01:19:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:00.181+0000 7f4424600640 1 -- 172.21.15.27:0/891182334 wait complete. 2024-09-21T01:19:00.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:00 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/891182334' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T01:19:00.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:00 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/891182334' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T01:19:00.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:00 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/891182334' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T01:19:01.059 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T01:19:01.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T01:19:01.444 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:01.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:01 smithi027 ceph-mon[30375]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:01.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:01.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:01.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:19:01.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:19:01.891 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:01 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:01 smithi089 ceph-mon[34486]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:19:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:19:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:01 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.076+0000 7f3e74e00640 1 -- 172.21.15.27:0/865544942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 msgr2=0x7f3e7010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.076+0000 7f3e74e00640 1 --2- 172.21.15.27:0/865544942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 0x7f3e7010d660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009900 tx=0x7f3e5c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 -- 172.21.15.27:0/865544942 shutdown_connections 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 --2- 172.21.15.27:0/865544942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e7010dc20 0x7f3e70110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 --2- 172.21.15.27:0/865544942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 0x7f3e7010d660 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 --2- 172.21.15.27:0/865544942 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e70104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 -- 172.21.15.27:0/865544942 >> 172.21.15.27:0/865544942 conn(0x7f3e700fe2c0 msgr2=0x7f3e70100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 -- 172.21.15.27:0/865544942 shutdown_connections 2024-09-21T01:19:02.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.077+0000 7f3e74e00640 1 -- 172.21.15.27:0/865544942 wait complete. 2024-09-21T01:19:02.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 Processor -- start 2024-09-21T01:19:02.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 -- start start 2024-09-21T01:19:02.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e7019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 0x7f3e7019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e7010dc20 0x7f3e701a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3e701138d0 con 0x7f3e70105110 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70113750 con 0x7f3e701047c0 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.078+0000 7f3e74e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70113a50 con 0x7f3e7010dc20 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e7019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e7019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:47324/0 (socket says 172.21.15.27:47324) 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e7010dc20 0x7f3e701a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 -- 172.21.15.27:0/156012555 learned_addr learned my addr 172.21.15.27:0/156012555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6ea00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 0x7f3e7019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 -- 172.21.15.27:0/156012555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e7010dc20 msgr2=0x7f3e701a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e7010dc20 0x7f3e701a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 -- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 msgr2=0x7f3e7019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:02.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 0x7f3e7019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 -- 172.21.15.27:0/156012555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e701a2430 con 0x7f3e701047c0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.079+0000 7f3e6f400640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e7019c4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000e9f0 tx=0x7f3e6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.080+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000cd80 con 0x7f3e701047c0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.080+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e600108e0 con 0x7f3e701047c0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.080+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e701a2720 con 0x7f3e701047c0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.080+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60018980 con 0x7f3e701047c0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.080+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e701a8e80 con 0x7f3e701047c0 2024-09-21T01:19:02.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.081+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e70104bd0 con 0x7f3e701047c0 2024-09-21T01:19:02.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.083+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3e60010430 con 0x7f3e701047c0 2024-09-21T01:19:02.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.085+0000 7f3e67e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3e540b6560 0x7f3e540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:02.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.085+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f3e600e77f0 con 0x7f3e701047c0 2024-09-21T01:19:02.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.085+0000 7f3e6ea00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3e540b6560 0x7f3e540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:02.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.086+0000 7f3e6ea00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3e540b6560 0x7f3e540b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02a9e0 tx=0x7f3e5c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:02.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.087+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e600b2790 con 0x7f3e701047c0 2024-09-21T01:19:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:01 smithi098 ceph-mon[34372]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:19:02.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:19:02.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:01 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:19:02.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.289+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3e701a9170 con 0x7f3e701047c0 2024-09-21T01:19:02.672 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-09-21T01:19:02.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.668+0000 7f3e67e00640 1 -- 172.21.15.27:0/156012555 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f3e600b76d0 con 0x7f3e701047c0 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3e540b6560 msgr2=0x7f3e540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3e540b6560 0x7f3e540b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02a9e0 tx=0x7f3e5c005a20 comp rx=0 tx=0).stop 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 msgr2=0x7f3e7019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e7019c4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000e9f0 tx=0x7f3e6000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 shutdown_connections 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3e540b6560 0x7f3e540b8a70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e7010dc20 0x7f3e701a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e70105110 0x7f3e7019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 --2- 172.21.15.27:0/156012555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3e701047c0 0x7f3e7019c4c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:02.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.674+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 >> 172.21.15.27:0/156012555 conn(0x7f3e700fe2c0 msgr2=0x7f3e70102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:02.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.675+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 shutdown_connections 2024-09-21T01:19:02.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:02.675+0000 7f3e74e00640 1 -- 172.21.15.27:0/156012555 wait complete. 2024-09-21T01:19:02.847 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:02 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/156012555' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:19:02.848 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:02 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:19:02.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:02 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/156012555' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:19:03.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:02 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:19:03.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:02 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/156012555' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:19:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:02 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T01:19:03.611 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T01:19:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:03 smithi027 ceph-mon[30375]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:03 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T01:19:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:03 smithi027 ceph-mon[30375]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T01:19:04.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:03 smithi089 ceph-mon[34486]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:04.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:03 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T01:19:04.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:03 smithi089 ceph-mon[34486]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T01:19:04.016 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:03 smithi098 ceph-mon[34372]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:03 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T01:19:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:03 smithi098 ceph-mon[34372]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T01:19:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 -- 172.21.15.27:0/2455116034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8240105bc0 msgr2=0x7f8240105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 --2- 172.21.15.27:0/2455116034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8240105bc0 0x7f8240105fd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8234009860 tx=0x7f823402a340 comp rx=0 tx=0).stop 2024-09-21T01:19:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 -- 172.21.15.27:0/2455116034 shutdown_connections 2024-09-21T01:19:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 --2- 172.21.15.27:0/2455116034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010be50 0x7f8240112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:04.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 --2- 172.21.15.27:0/2455116034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f824010b500 0x7f824010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 --2- 172.21.15.27:0/2455116034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8240105bc0 0x7f8240105fd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 -- 172.21.15.27:0/2455116034 >> 172.21.15.27:0/2455116034 conn(0x7f8240101670 msgr2=0x7f8240103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 -- 172.21.15.27:0/2455116034 shutdown_connections 2024-09-21T01:19:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.717+0000 7f8246600640 1 -- 172.21.15.27:0/2455116034 wait complete. 2024-09-21T01:19:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.718+0000 7f8246600640 1 Processor -- start 2024-09-21T01:19:04.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 -- start start 2024-09-21T01:19:04.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8240105bc0 0x7f824019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:04.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 0x7f824019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:04.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f824010be50 0x7f82401a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8240115b10 con 0x7f824010b500 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8240115990 con 0x7f8240105bc0 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8246600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8240115c90 con 0x7f824010be50 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f823fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 0x7f824019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f823fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 0x7f824019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55750/0 (socket says 172.21.15.27:55750) 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f8245200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8240105bc0 0x7f824019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.719+0000 7f823fe00640 1 -- 172.21.15.27:0/3355834726 learned_addr learned my addr 172.21.15.27:0/3355834726 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245c00640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f824010be50 0x7f82401a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245200640 1 -- 172.21.15.27:0/3355834726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f824010be50 msgr2=0x7f82401a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245200640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f824010be50 0x7f82401a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245200640 1 -- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 msgr2=0x7f824019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245200640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 0x7f824019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245200640 1 -- 172.21.15.27:0/3355834726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82401a2490 con 0x7f8240105bc0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f823fe00640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 0x7f824019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.720+0000 7f8245200640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8240105bc0 0x7f824019c520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8234035040 tx=0x7f8234032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.721+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234040020 con 0x7f8240105bc0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.721+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234030e10 con 0x7f8240105bc0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.721+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234037040 con 0x7f8240105bc0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.721+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82401a2720 con 0x7f8240105bc0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.721+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82401a8ee0 con 0x7f8240105bc0 2024-09-21T01:19:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.722+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8240106e40 con 0x7f8240105bc0 2024-09-21T01:19:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.724+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8234043020 con 0x7f8240105bc0 2024-09-21T01:19:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.726+0000 7f823d600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f82280b6560 0x7f82280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.726+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7712+0+0 (secure 0 0 0) 0x7f82341073d0 con 0x7f8240105bc0 2024-09-21T01:19:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.726+0000 7f823fe00640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f82280b6560 0x7f82280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.727+0000 7f823fe00640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f82280b6560 0x7f82280b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f824010afb0 tx=0x7f822c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:04.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.728+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f823404a020 con 0x7f8240105bc0 2024-09-21T01:19:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:04.928+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f8240105fd0 con 0x7f8240105bc0 2024-09-21T01:19:04.929 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:04 smithi027 ceph-mon[30375]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T01:19:04.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:04 smithi089 ceph-mon[34486]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T01:19:05.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:04 smithi098 ceph-mon[34372]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T01:19:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.692+0000 7f823d600640 1 -- 172.21.15.27:0/3355834726 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f82340cdec0 con 0x7f8240105bc0 2024-09-21T01:19:05.693 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-09-21T01:19:05.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.698+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f82280b6560 msgr2=0x7f82280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.698+0000 7f8246600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f82280b6560 0x7f82280b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f824010afb0 tx=0x7f822c005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.698+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8240105bc0 msgr2=0x7f824019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.698+0000 7f8246600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8240105bc0 0x7f824019c520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8234035040 tx=0x7f8234032d50 comp rx=0 tx=0).stop 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.699+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 shutdown_connections 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.699+0000 7f8246600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f82280b6560 0x7f82280b8a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.699+0000 7f8246600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f824010be50 0x7f82401a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.699+0000 7f8246600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f824010b500 0x7f824019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:05.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.699+0000 7f8246600640 1 --2- 172.21.15.27:0/3355834726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8240105bc0 0x7f824019c520 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8234035040 tx=0x7f8234032d50 comp rx=0 tx=0).stop 2024-09-21T01:19:05.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.699+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 >> 172.21.15.27:0/3355834726 conn(0x7f8240101670 msgr2=0x7f8240103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:05.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.700+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 shutdown_connections 2024-09-21T01:19:05.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:05.700+0000 7f8246600640 1 -- 172.21.15.27:0/3355834726 wait complete. 2024-09-21T01:19:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:05 smithi089 ceph-mon[34486]: pgmap v284: 33 pgs: 26 unknown, 7 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:05 smithi089 ceph-mon[34486]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T01:19:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:05 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3355834726' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T01:19:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:05 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T01:19:06.109 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:05 smithi027 ceph-mon[30375]: pgmap v284: 33 pgs: 26 unknown, 7 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:06.110 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:05 smithi027 ceph-mon[30375]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T01:19:06.110 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:05 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3355834726' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T01:19:06.110 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:05 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T01:19:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:05 smithi098 ceph-mon[34372]: pgmap v284: 33 pgs: 26 unknown, 7 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:05 smithi098 ceph-mon[34372]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T01:19:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:05 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3355834726' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T01:19:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:05 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T01:19:06.583 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T01:19:06.989 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:07.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:06 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T01:19:07.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:06 smithi089 ceph-mon[34486]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T01:19:07.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:06 smithi089 ceph-mon[34486]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T01:19:07.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:06 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T01:19:07.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:06 smithi027 ceph-mon[30375]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T01:19:07.129 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:06 smithi027 ceph-mon[30375]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T01:19:07.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:06 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T01:19:07.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:06 smithi098 ceph-mon[34372]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T01:19:07.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:06 smithi098 ceph-mon[34372]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T01:19:07.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 -- 172.21.15.27:0/1503462208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a0105db0 msgr2=0x7f03a01061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:07.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 --2- 172.21.15.27:0/1503462208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a0105db0 0x7f03a01061c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0390009900 tx=0x7f039002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 -- 172.21.15.27:0/1503462208 shutdown_connections 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 --2- 172.21.15.27:0/1503462208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a010c040 0x7f03a0112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 --2- 172.21.15.27:0/1503462208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a010b6f0 0x7f03a010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 --2- 172.21.15.27:0/1503462208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a0105db0 0x7f03a01061c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 -- 172.21.15.27:0/1503462208 >> 172.21.15.27:0/1503462208 conn(0x7f03a01017e0 msgr2=0x7f03a0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 -- 172.21.15.27:0/1503462208 shutdown_connections 2024-09-21T01:19:07.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.681+0000 7f03a6000640 1 -- 172.21.15.27:0/1503462208 wait complete. 2024-09-21T01:19:07.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.682+0000 7f03a6000640 1 Processor -- start 2024-09-21T01:19:07.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 -- start start 2024-09-21T01:19:07.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 0x7f03a019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:07.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a010b6f0 0x7f03a019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:07.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a010c040 0x7f03a01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0115d50 con 0x7f03a010c040 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0115bd0 con 0x7f03a0105db0 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f03a6000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0115ed0 con 0x7f03a010b6f0 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f039f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 0x7f03a019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f039f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 0x7f03a019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:55444/0 (socket says 172.21.15.27:55444) 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f039f400640 1 -- 172.21.15.27:0/3281017530 learned_addr learned my addr 172.21.15.27:0/3281017530 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f0397e00640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a010b6f0 0x7f03a019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.683+0000 7f039fe00640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a010c040 0x7f03a01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039f400640 1 -- 172.21.15.27:0/3281017530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a010b6f0 msgr2=0x7f03a019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039f400640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a010b6f0 0x7f03a019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039f400640 1 -- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a010c040 msgr2=0x7f03a01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039f400640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a010c040 0x7f03a01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039f400640 1 -- 172.21.15.27:0/3281017530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a01a26d0 con 0x7f03a0105db0 2024-09-21T01:19:07.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039f400640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 0x7f03a019c760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0390005ab0 tx=0x7f0390003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:07.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039003f070 con 0x7f03a0105db0 2024-09-21T01:19:07.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.684+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a01a2960 con 0x7f03a0105db0 2024-09-21T01:19:07.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.685+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0390031b60 con 0x7f03a0105db0 2024-09-21T01:19:07.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.685+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390035030 con 0x7f03a0105db0 2024-09-21T01:19:07.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.685+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a01a9120 con 0x7f03a0105db0 2024-09-21T01:19:07.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.686+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a01061c0 con 0x7f03a0105db0 2024-09-21T01:19:07.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.688+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0390042070 con 0x7f03a0105db0 2024-09-21T01:19:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.690+0000 7f039cc00640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f03840b65e0 0x7f03840b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.690+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8094+0+0 (secure 0 0 0) 0x7f0390108150 con 0x7f03a0105db0 2024-09-21T01:19:07.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.690+0000 7f0397e00640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f03840b65e0 0x7f03840b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:07.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.691+0000 7f0397e00640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f03840b65e0 0x7f03840b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03a0104fc0 tx=0x7f0388005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.692+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03900d2e00 con 0x7f03a0105db0 2024-09-21T01:19:07.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:07.933+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f03a010de10 con 0x7f03a0105db0 2024-09-21T01:19:07.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:07 smithi027 ceph-mon[30375]: pgmap v287: 97 pgs: 81 unknown, 16 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:07.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:07 smithi027 ceph-mon[30375]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T01:19:07.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:07 smithi089 ceph-mon[34486]: pgmap v287: 97 pgs: 81 unknown, 16 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:08.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:07 smithi089 ceph-mon[34486]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T01:19:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:07 smithi098 ceph-mon[34372]: pgmap v287: 97 pgs: 81 unknown, 16 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:07 smithi098 ceph-mon[34372]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T01:19:08.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.733+0000 7f039cc00640 1 -- 172.21.15.27:0/3281017530 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f03900d7d40 con 0x7f03a0105db0 2024-09-21T01:19:08.734 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T01:19:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.737+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f03840b65e0 msgr2=0x7f03840b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.737+0000 7f03a6000640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f03840b65e0 0x7f03840b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03a0104fc0 tx=0x7f0388005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.737+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 msgr2=0x7f03a019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:08.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.737+0000 7f03a6000640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 0x7f03a019c760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0390005ab0 tx=0x7f0390003c10 comp rx=0 tx=0).stop 2024-09-21T01:19:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.738+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 shutdown_connections 2024-09-21T01:19:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.738+0000 7f03a6000640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f03840b65e0 0x7f03840b8af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.738+0000 7f03a6000640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a010c040 0x7f03a01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.738+0000 7f03a6000640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03a010b6f0 0x7f03a019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.738+0000 7f03a6000640 1 --2- 172.21.15.27:0/3281017530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a0105db0 0x7f03a019c760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.738+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 >> 172.21.15.27:0/3281017530 conn(0x7f03a01017e0 msgr2=0x7f03a010ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.739+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 shutdown_connections 2024-09-21T01:19:08.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:08.739+0000 7f03a6000640 1 -- 172.21.15.27:0/3281017530 wait complete. 2024-09-21T01:19:09.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:08 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3281017530' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:19:09.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:08 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:19:09.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:08 smithi089 ceph-mon[34486]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T01:19:09.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:08 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3281017530' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:19:09.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:08 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:19:09.122 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:08 smithi027 ceph-mon[30375]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T01:19:09.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:08 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3281017530' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:19:09.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:08 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T01:19:09.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:08 smithi098 ceph-mon[34372]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T01:19:09.628 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T01:19:09.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T01:19:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:09 smithi027 ceph-mon[30375]: pgmap v290: 97 pgs: 4 creating+peering, 30 unknown, 63 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:09 smithi027 ceph-mon[30375]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T01:19:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:09 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T01:19:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:09 smithi027 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T01:19:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:09 smithi027 ceph-mon[30375]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T01:19:10.000 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:19:09 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: 2024-09-21T01:19:09.854+0000 7fe142be0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T01:19:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:09 smithi089 ceph-mon[34486]: pgmap v290: 97 pgs: 4 creating+peering, 30 unknown, 63 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:09 smithi089 ceph-mon[34486]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T01:19:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:09 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T01:19:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:09 smithi089 ceph-mon[34486]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T01:19:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:09 smithi089 ceph-mon[34486]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T01:19:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:09 smithi098 ceph-mon[34372]: pgmap v290: 97 pgs: 4 creating+peering, 30 unknown, 63 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:09 smithi098 ceph-mon[34372]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T01:19:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:09 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T01:19:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:09 smithi098 ceph-mon[34372]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T01:19:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:09 smithi098 ceph-mon[34372]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.674+0000 7fcdb2000640 1 -- 172.21.15.27:0/2738370581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 msgr2=0x7fcdac10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.674+0000 7fcdb2000640 1 --2- 172.21.15.27:0/2738370581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac10b910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009900 tx=0x7fcd9402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 -- 172.21.15.27:0/2738370581 shutdown_connections 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 --2- 172.21.15.27:0/2738370581 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdac10be50 0x7fcdac112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 --2- 172.21.15.27:0/2738370581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac10b910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 --2- 172.21.15.27:0/2738370581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac105bc0 0x7fcdac105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 -- 172.21.15.27:0/2738370581 >> 172.21.15.27:0/2738370581 conn(0x7fcdac101630 msgr2=0x7fcdac103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 -- 172.21.15.27:0/2738370581 shutdown_connections 2024-09-21T01:19:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.675+0000 7fcdb2000640 1 -- 172.21.15.27:0/2738370581 wait complete. 2024-09-21T01:19:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 Processor -- start 2024-09-21T01:19:10.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 -- start start 2024-09-21T01:19:10.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdac105bc0 0x7fcdac19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:10.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:10.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac10be50 0x7fcdac1a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:10.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac115b10 con 0x7fcdac10b500 2024-09-21T01:19:10.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac115990 con 0x7fcdac105bc0 2024-09-21T01:19:10.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.676+0000 7fcdb2000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac115c90 con 0x7fcdac10be50 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb1600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac10be50 0x7fcdac1a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36470/0 (socket says 172.21.15.27:36470) 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcda3e00640 1 -- 172.21.15.27:0/1887830054 learned_addr learned my addr 172.21.15.27:0/1887830054 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb0c00640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdac105bc0 0x7fcdac19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb1600640 1 -- 172.21.15.27:0/1887830054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdac105bc0 msgr2=0x7fcdac19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb1600640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdac105bc0 0x7fcdac19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:10.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb1600640 1 -- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 msgr2=0x7fcdac19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:10.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb1600640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:10.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcdb1600640 1 -- 172.21.15.27:0/1887830054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac1a24f0 con 0x7fcdac10be50 2024-09-21T01:19:10.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.677+0000 7fcda3e00640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:10.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.678+0000 7fcdb1600640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac10be50 0x7fcdac1a1dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcda400da10 tx=0x7fcda400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.678+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda400bd70 con 0x7fcdac10be50 2024-09-21T01:19:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.678+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda4003e20 con 0x7fcdac10be50 2024-09-21T01:19:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.678+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac1a27e0 con 0x7fcdac10be50 2024-09-21T01:19:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.678+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40106c0 con 0x7fcdac10be50 2024-09-21T01:19:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.678+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac1a8f40 con 0x7fcdac10be50 2024-09-21T01:19:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.679+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdac105fd0 con 0x7fcdac10be50 2024-09-21T01:19:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.682+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fcda4004270 con 0x7fcdac10be50 2024-09-21T01:19:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.683+0000 7fcdaa000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcd900b6560 0x7fcd900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.684+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7fcda40e7a40 con 0x7fcdac10be50 2024-09-21T01:19:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.684+0000 7fcdb0c00640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcd900b6560 0x7fcd900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.684+0000 7fcdb0c00640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcd900b6560 0x7fcd900b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009bd0 tx=0x7fcd9c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.685+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcda40ae290 con 0x7fcdac10be50 2024-09-21T01:19:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:10.900+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fcdac19d6b0 con 0x7fcdac10be50 2024-09-21T01:19:11.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:10 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] up:active 2024-09-21T01:19:11.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:10 smithi089 ceph-mon[34486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:11.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:10 smithi089 ceph-mon[34486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:11.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:10 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] up:active 2024-09-21T01:19:11.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:10 smithi098 ceph-mon[34372]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:11.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:10 smithi098 ceph-mon[34372]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:10 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] up:active 2024-09-21T01:19:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:10 smithi027 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:10 smithi027 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:11.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.760+0000 7fcdaa000640 1 -- 172.21.15.27:0/1887830054 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7fcdac19d6b0 con 0x7fcdac10be50 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcd900b6560 msgr2=0x7fcd900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcd900b6560 0x7fcd900b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009bd0 tx=0x7fcd9c005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac10be50 msgr2=0x7fcdac1a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac10be50 0x7fcdac1a1dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcda400da10 tx=0x7fcda400ded0 comp rx=0 tx=0).stop 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 shutdown_connections 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcd900b6560 0x7fcd900b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdac10be50 0x7fcdac1a1dc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdac10b500 0x7fcdac19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 --2- 172.21.15.27:0/1887830054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcdac105bc0 0x7fcdac19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 >> 172.21.15.27:0/1887830054 conn(0x7fcdac101630 msgr2=0x7fcdac103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:11.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 shutdown_connections 2024-09-21T01:19:11.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:11.764+0000 7fcdb2000640 1 -- 172.21.15.27:0/1887830054 wait complete. 2024-09-21T01:19:12.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:11 smithi027 ceph-mon[30375]: pgmap v292: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 866 B/s wr, 3 op/s 2024-09-21T01:19:12.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:11 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1887830054' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T01:19:12.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:11 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T01:19:12.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:11 smithi098 ceph-mon[34372]: pgmap v292: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 866 B/s wr, 3 op/s 2024-09-21T01:19:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:11 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1887830054' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T01:19:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:11 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T01:19:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:11 smithi089 ceph-mon[34486]: pgmap v292: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 866 B/s wr, 3 op/s 2024-09-21T01:19:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:11 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1887830054' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T01:19:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:11 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T01:19:12.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T01:19:12.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:12 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T01:19:12.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:12 smithi027 ceph-mon[30375]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T01:19:13.049 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:12 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T01:19:13.049 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:12 smithi098 ceph-mon[34372]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T01:19:13.089 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:12 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T01:19:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:12 smithi089 ceph-mon[34486]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T01:19:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 -- 172.21.15.27:0/2811997806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10b6c0 msgr2=0x7f126c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 --2- 172.21.15.27:0/2811997806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10b6c0 0x7f126c10bad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1254009900 tx=0x7f125402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 -- 172.21.15.27:0/2811997806 shutdown_connections 2024-09-21T01:19:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 --2- 172.21.15.27:0/2811997806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c10c010 0x7f126c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 --2- 172.21.15.27:0/2811997806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10b6c0 0x7f126c10bad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 --2- 172.21.15.27:0/2811997806 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f126c105d80 0x7f126c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 -- 172.21.15.27:0/2811997806 >> 172.21.15.27:0/2811997806 conn(0x7f126c1017d0 msgr2=0x7f126c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 -- 172.21.15.27:0/2811997806 shutdown_connections 2024-09-21T01:19:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.720+0000 7f1273000640 1 -- 172.21.15.27:0/2811997806 wait complete. 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.721+0000 7f1273000640 1 Processor -- start 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 -- start start 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c105d80 0x7f126c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f126c10b6c0 0x7f126c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10c010 0x7f126c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f126c115cd0 con 0x7f126c105d80 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f126c115b50 con 0x7f126c10b6c0 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1273000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f126c115e50 con 0x7f126c10c010 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f126be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c105d80 0x7f126c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1271000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10c010 0x7f126c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:13.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f126be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c105d80 0x7f126c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36494/0 (socket says 172.21.15.27:36494) 2024-09-21T01:19:13.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f126be00640 1 -- 172.21.15.27:0/3513029916 learned_addr learned my addr 172.21.15.27:0/3513029916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:13.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.722+0000 7f1263400640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f126c10b6c0 0x7f126c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:13.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1271000640 1 -- 172.21.15.27:0/3513029916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f126c10b6c0 msgr2=0x7f126c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1271000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f126c10b6c0 0x7f126c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1271000640 1 -- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c105d80 msgr2=0x7f126c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1271000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c105d80 0x7f126c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1271000640 1 -- 172.21.15.27:0/3513029916 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c1a25d0 con 0x7f126c10c010 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1271000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10c010 0x7f126c1a1ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1264009fd0 tx=0x7f126400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126400ce20 con 0x7f126c10c010 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1264005020 con 0x7f126c10c010 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264010690 con 0x7f126c10c010 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.723+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c1a28c0 con 0x7f126c10c010 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.724+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c1a9020 con 0x7f126c10c010 2024-09-21T01:19:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.725+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c106190 con 0x7f126c10c010 2024-09-21T01:19:13.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.727+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f12640054d0 con 0x7f126c10c010 2024-09-21T01:19:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.728+0000 7f1269600640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f12500b6560 0x7f12500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.729+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8492+0+0 (secure 0 0 0) 0x7f12640e7af0 con 0x7f126c10c010 2024-09-21T01:19:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.729+0000 7f126be00640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f12500b6560 0x7f12500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.730+0000 7f126be00640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f12500b6560 0x7f12500b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f125c005fd0 tx=0x7f125c00b4a0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.730+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12640ae220 con 0x7f126c10c010 2024-09-21T01:19:13.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:13 smithi027 ceph-mon[30375]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-21T01:19:13.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:13.935+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f126c19d790 con 0x7f126c10c010 2024-09-21T01:19:14.152 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:13 smithi098 ceph-mon[34372]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-21T01:19:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:13 smithi089 ceph-mon[34486]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-21T01:19:15.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:14 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3513029916' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:19:15.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:14 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:19:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:14 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3513029916' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:19:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:14 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:19:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:14 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3513029916' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:19:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:14 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T01:19:15.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.787+0000 7f1269600640 1 -- 172.21.15.27:0/3513029916 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f126c19d790 con 0x7f126c10c010 2024-09-21T01:19:15.790 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' created 2024-09-21T01:19:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.795+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f12500b6560 msgr2=0x7f12500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.795+0000 7f1273000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f12500b6560 0x7f12500b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f125c005fd0 tx=0x7f125c00b4a0 comp rx=0 tx=0).stop 2024-09-21T01:19:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.795+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10c010 msgr2=0x7f126c1a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.795+0000 7f1273000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10c010 0x7f126c1a1ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1264009fd0 tx=0x7f126400ef10 comp rx=0 tx=0).stop 2024-09-21T01:19:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.796+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 shutdown_connections 2024-09-21T01:19:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.796+0000 7f1273000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f12500b6560 0x7f12500b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.796+0000 7f1273000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f126c10c010 0x7f126c1a1ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.796+0000 7f1273000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f126c10b6c0 0x7f126c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.796+0000 7f1273000640 1 --2- 172.21.15.27:0/3513029916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c105d80 0x7f126c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:15.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.796+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 >> 172.21.15.27:0/3513029916 conn(0x7f126c1017d0 msgr2=0x7f126c103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.797+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 shutdown_connections 2024-09-21T01:19:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:15.798+0000 7f1273000640 1 -- 172.21.15.27:0/3513029916 wait complete. 2024-09-21T01:19:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:15 smithi098 ceph-mon[34372]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T01:19:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:15 smithi098 ceph-mon[34372]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T01:19:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:15 smithi027 ceph-mon[30375]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T01:19:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:15 smithi027 ceph-mon[30375]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T01:19:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:15 smithi089 ceph-mon[34486]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T01:19:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:15 smithi089 ceph-mon[34486]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T01:19:16.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T01:19:16.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:16 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T01:19:16.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:16 smithi027 ceph-mon[30375]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T01:19:17.113 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:17.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:16 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T01:19:17.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:16 smithi098 ceph-mon[34372]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T01:19:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:16 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T01:19:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:16 smithi089 ceph-mon[34486]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T01:19:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 -- 172.21.15.27:0/825192142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 msgr2=0x7f99e0075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 --2- 172.21.15.27:0/825192142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e0075b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009900 tx=0x7f99d002a410 comp rx=0 tx=0).stop 2024-09-21T01:19:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 -- 172.21.15.27:0/825192142 shutdown_connections 2024-09-21T01:19:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 --2- 172.21.15.27:0/825192142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 --2- 172.21.15.27:0/825192142 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99e0076080 0x7f99e010fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 --2- 172.21.15.27:0/825192142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e0075b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 -- 172.21.15.27:0/825192142 >> 172.21.15.27:0/825192142 conn(0x7f99e00ff400 msgr2=0x7f99e0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 -- 172.21.15.27:0/825192142 shutdown_connections 2024-09-21T01:19:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.805+0000 7f99df200640 1 -- 172.21.15.27:0/825192142 wait complete. 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 Processor -- start 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 -- start start 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99e0076080 0x7f99e019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0115ad0 con 0x7f99e0075730 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0115950 con 0x7f99e0076080 2024-09-21T01:19:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.807+0000 7f99df200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0115c50 con 0x7f99e010ff90 2024-09-21T01:19:17.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:17.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99dde00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:17.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:58672/0 (socket says 172.21.15.27:58672) 2024-09-21T01:19:17.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 -- 172.21.15.27:0/3903070793 learned_addr learned my addr 172.21.15.27:0/3903070793 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:17.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99dd400640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99e0076080 0x7f99e019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:17.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 -- 172.21.15.27:0/3903070793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99e0076080 msgr2=0x7f99e019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99e0076080 0x7f99e019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 -- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 msgr2=0x7f99e019c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e019c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99de800640 1 -- 172.21.15.27:0/3903070793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e01a2410 con 0x7f99e010ff90 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.808+0000 7f99dde00640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e019c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.809+0000 7f99de800640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01a1ce0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99d800eb10 tx=0x7f99d800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.809+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8004b20 con 0x7f99e010ff90 2024-09-21T01:19:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.809+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d8004cc0 con 0x7f99e010ff90 2024-09-21T01:19:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.809+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01a2700 con 0x7f99e010ff90 2024-09-21T01:19:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.809+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8010670 con 0x7f99e010ff90 2024-09-21T01:19:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.809+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01a8e60 con 0x7f99e010ff90 2024-09-21T01:19:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.810+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e0075b40 con 0x7f99e010ff90 2024-09-21T01:19:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.812+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f99d8005020 con 0x7f99e010ff90 2024-09-21T01:19:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.814+0000 7f99d6a00640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99c40b65e0 0x7f99c40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.814+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8985+0+0 (secure 0 0 0) 0x7f99d80e7c10 con 0x7f99e010ff90 2024-09-21T01:19:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.814+0000 7f99dde00640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99c40b65e0 0x7f99c40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.815+0000 7f99dde00640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99c40b65e0 0x7f99c40b8af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99d0031440 tx=0x7f99d0032ed0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:17.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:17.818+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99d80ae150 con 0x7f99e010ff90 2024-09-21T01:19:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:17 smithi027 ceph-mon[30375]: pgmap v298: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-21T01:19:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:17 smithi027 ceph-mon[30375]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T01:19:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.018+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f99e019d5d0 con 0x7f99e010ff90 2024-09-21T01:19:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:17 smithi098 ceph-mon[34372]: pgmap v298: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-21T01:19:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:17 smithi098 ceph-mon[34372]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T01:19:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:17 smithi089 ceph-mon[34486]: pgmap v298: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 4 op/s 2024-09-21T01:19:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:17 smithi089 ceph-mon[34486]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T01:19:18.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.810+0000 7f99d6a00640 1 -- 172.21.15.27:0/3903070793 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f99d80b2530 con 0x7f99e010ff90 2024-09-21T01:19:18.811 INFO:teuthology.orchestra.run.smithi027.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T01:19:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99c40b65e0 msgr2=0x7f99c40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99c40b65e0 0x7f99c40b8af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99d0031440 tx=0x7f99d0032ed0 comp rx=0 tx=0).stop 2024-09-21T01:19:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 msgr2=0x7f99e01a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01a1ce0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99d800eb10 tx=0x7f99d800c2d0 comp rx=0 tx=0).stop 2024-09-21T01:19:18.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 shutdown_connections 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99c40b65e0 0x7f99c40b8af0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e010ff90 0x7f99e01a1ce0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99e0076080 0x7f99e019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 --2- 172.21.15.27:0/3903070793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e0075730 0x7f99e019c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.813+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 >> 172.21.15.27:0/3903070793 conn(0x7f99e00ff400 msgr2=0x7f99e0101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.814+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 shutdown_connections 2024-09-21T01:19:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:18.814+0000 7f99df200640 1 -- 172.21.15.27:0/3903070793 wait complete. 2024-09-21T01:19:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:18 smithi098 ceph-mon[34372]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T01:19:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:18 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3903070793' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T01:19:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:18 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T01:19:19.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:18 smithi027 ceph-mon[30375]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T01:19:19.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:18 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3903070793' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T01:19:19.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:18 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T01:19:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:18 smithi089 ceph-mon[34486]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T01:19:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:18 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3903070793' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T01:19:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:18 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T01:19:19.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T01:19:20.141 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:20.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:19 smithi098 ceph-mon[34372]: pgmap v301: 129 pgs: 2 creating+activating, 4 creating+peering, 17 unknown, 106 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:19 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T01:19:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:19 smithi098 ceph-mon[34372]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T01:19:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:19 smithi027 ceph-mon[30375]: pgmap v301: 129 pgs: 2 creating+activating, 4 creating+peering, 17 unknown, 106 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:19 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T01:19:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:19 smithi027 ceph-mon[30375]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T01:19:20.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:19 smithi089 ceph-mon[34486]: pgmap v301: 129 pgs: 2 creating+activating, 4 creating+peering, 17 unknown, 106 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:19 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T01:19:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:19 smithi089 ceph-mon[34486]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T01:19:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 -- 172.21.15.27:0/2636016957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 msgr2=0x7f3a1410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 --2- 172.21.15.27:0/2636016957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1410b910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009900 tx=0x7f3a0002a410 comp rx=0 tx=0).stop 2024-09-21T01:19:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 -- 172.21.15.27:0/2636016957 shutdown_connections 2024-09-21T01:19:20.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 --2- 172.21.15.27:0/2636016957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 0x7f3a14112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 --2- 172.21.15.27:0/2636016957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1410b910 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 --2- 172.21.15.27:0/2636016957 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a14105bc0 0x7f3a14105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 -- 172.21.15.27:0/2636016957 >> 172.21.15.27:0/2636016957 conn(0x7f3a14101670 msgr2=0x7f3a14103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 -- 172.21.15.27:0/2636016957 shutdown_connections 2024-09-21T01:19:20.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.757+0000 7f3a19e00640 1 -- 172.21.15.27:0/2636016957 wait complete. 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.758+0000 7f3a19e00640 1 Processor -- start 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.758+0000 7f3a19e00640 1 -- start start 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a14105bc0 0x7f3a1419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 0x7f3a141a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14115b10 con 0x7f3a1410b500 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14115990 con 0x7f3a14105bc0 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14115c90 con 0x7f3a1410be50 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a13e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45256/0 (socket says 172.21.15.27:45256) 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a19400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 0x7f3a141a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a13e00640 1 -- 172.21.15.27:0/4258145144 learned_addr learned my addr 172.21.15.27:0/4258145144 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.759+0000 7f3a18a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a14105bc0 0x7f3a1419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19400640 1 -- 172.21.15.27:0/4258145144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a14105bc0 msgr2=0x7f3a1419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19400640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a14105bc0 0x7f3a1419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19400640 1 -- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 msgr2=0x7f3a1419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19400640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19400640 1 -- 172.21.15.27:0/4258145144 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141a2470 con 0x7f3a1410be50 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19400640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 0x7f3a141a1d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009fd0 tx=0x7f3a0c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00ce20 con 0x7f3a1410be50 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c005020 con 0x7f3a1410be50 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c010690 con 0x7f3a1410be50 2024-09-21T01:19:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.760+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141a2760 con 0x7f3a1410be50 2024-09-21T01:19:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.761+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a8ec0 con 0x7f3a1410be50 2024-09-21T01:19:20.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.762+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a14105fd0 con 0x7f3a1410be50 2024-09-21T01:19:20.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.764+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3a0c0054d0 con 0x7f3a1410be50 2024-09-21T01:19:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.766+0000 7f3a11600640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39fc0b6560 0x7f39fc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.766+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8985+0+0 (secure 0 0 0) 0x7f3a0c0e7d50 con 0x7f3a1410be50 2024-09-21T01:19:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.766+0000 7f3a18a00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39fc0b6560 0x7f39fc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:20.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.767+0000 7f3a18a00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39fc0b6560 0x7f39fc0b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a080095f0 tx=0x7f3a08005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:20.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:20.767+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3a0c0ae290 con 0x7f3a1410be50 2024-09-21T01:19:21.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.012+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f3a1419d630 con 0x7f3a1410be50 2024-09-21T01:19:21.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.822+0000 7f3a11600640 1 -- 172.21.15.27:0/4258145144 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f3a1419d630 con 0x7f3a1410be50 2024-09-21T01:19:21.823 INFO:teuthology.orchestra.run.smithi027.stderr:added data pool 4 to fsmap 2024-09-21T01:19:21.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39fc0b6560 msgr2=0x7f39fc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39fc0b6560 0x7f39fc0b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a080095f0 tx=0x7f3a08005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 msgr2=0x7f3a141a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 0x7f3a141a1d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009fd0 tx=0x7f3a0c00ef10 comp rx=0 tx=0).stop 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 shutdown_connections 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39fc0b6560 0x7f39fc0b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3a1410be50 0x7f3a141a1d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a1410b500 0x7f3a1419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 --2- 172.21.15.27:0/4258145144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3a14105bc0 0x7f3a1419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:21.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 >> 172.21.15.27:0/4258145144 conn(0x7f3a14101670 msgr2=0x7f3a1410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:21.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 shutdown_connections 2024-09-21T01:19:21.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:21.826+0000 7f3a19e00640 1 -- 172.21.15.27:0/4258145144 wait complete. 2024-09-21T01:19:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:21 smithi098 ceph-mon[34372]: pgmap v303: 129 pgs: 2 creating+activating, 4 creating+peering, 123 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4258145144' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T01:19:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:21 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T01:19:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:21 smithi027 ceph-mon[30375]: pgmap v303: 129 pgs: 2 creating+activating, 4 creating+peering, 123 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4258145144' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T01:19:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:21 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T01:19:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:21 smithi089 ceph-mon[34486]: pgmap v303: 129 pgs: 2 creating+activating, 4 creating+peering, 123 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4258145144' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T01:19:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:21 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T01:19:22.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:19:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:22 smithi098 ceph-mon[34372]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T01:19:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:22 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T01:19:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:22 smithi098 ceph-mon[34372]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:22 smithi027 ceph-mon[30375]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T01:19:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:22 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T01:19:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:22 smithi027 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:23.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:22 smithi089 ceph-mon[34486]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T01:19:23.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:22 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T01:19:23.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:22 smithi089 ceph-mon[34486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:19:23.255 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 -- 172.21.15.27:0/385673898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e241039b0 msgr2=0x7f6e24103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 --2- 172.21.15.27:0/385673898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e241039b0 0x7f6e24103dc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009900 tx=0x7f6e1402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 -- 172.21.15.27:0/385673898 shutdown_connections 2024-09-21T01:19:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 --2- 172.21.15.27:0/385673898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e24109c40 0x7f6e24110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:23.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 --2- 172.21.15.27:0/385673898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e241092f0 0x7f6e24109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 --2- 172.21.15.27:0/385673898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e241039b0 0x7f6e24103dc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 -- 172.21.15.27:0/385673898 >> 172.21.15.27:0/385673898 conn(0x7f6e240ff400 msgr2=0x7f6e24101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 -- 172.21.15.27:0/385673898 shutdown_connections 2024-09-21T01:19:23.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.908+0000 7f6e29800640 1 -- 172.21.15.27:0/385673898 wait complete. 2024-09-21T01:19:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.909+0000 7f6e29800640 1 Processor -- start 2024-09-21T01:19:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.909+0000 7f6e29800640 1 -- start start 2024-09-21T01:19:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e29800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 0x7f6e2419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e29800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e241092f0 0x7f6e2419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e29800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e24109c40 0x7f6e2419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:23.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e29800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6e24113950 con 0x7f6e241039b0 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e29800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e241137d0 con 0x7f6e24109c40 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e29800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e24113ad0 con 0x7f6e241092f0 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e23e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 0x7f6e2419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e23e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 0x7f6e2419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45262/0 (socket says 172.21.15.27:45262) 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e23e00640 1 -- 172.21.15.27:0/916701612 learned_addr learned my addr 172.21.15.27:0/916701612 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e23400640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e241092f0 0x7f6e2419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:23.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.910+0000 7f6e28e00640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e24109c40 0x7f6e2419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e23e00640 1 -- 172.21.15.27:0/916701612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e241092f0 msgr2=0x7f6e2419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e23e00640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e241092f0 0x7f6e2419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e23e00640 1 -- 172.21.15.27:0/916701612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e24109c40 msgr2=0x7f6e2419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e23e00640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e24109c40 0x7f6e2419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e23e00640 1 -- 172.21.15.27:0/916701612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e241a0250 con 0x7f6e241039b0 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e23e00640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 0x7f6e2419a2e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e14005ab0 tx=0x7f6e14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1403f070 con 0x7f6e241039b0 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e241a04e0 con 0x7f6e241039b0 2024-09-21T01:19:23.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e14031b60 con 0x7f6e241039b0 2024-09-21T01:19:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14035030 con 0x7f6e241039b0 2024-09-21T01:19:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.911+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e241a6ca0 con 0x7f6e241039b0 2024-09-21T01:19:23.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.913+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e24103dc0 con 0x7f6e241039b0 2024-09-21T01:19:23.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.915+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f6e14042070 con 0x7f6e241039b0 2024-09-21T01:19:23.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.917+0000 7f6e20c00640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6e080b6560 0x7f6e080b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:23.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.917+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f6e14108480 con 0x7f6e241039b0 2024-09-21T01:19:23.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.917+0000 7f6e23400640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6e080b6560 0x7f6e080b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:23.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.918+0000 7f6e23400640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6e080b6560 0x7f6e080b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e24108d70 tx=0x7f6e10005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:23.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:23.919+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e140d2d80 con 0x7f6e241039b0 2024-09-21T01:19:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:23 smithi098 ceph-mon[34372]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:24.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.157+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e2410ba40 con 0x7f6e241039b0 2024-09-21T01:19:24.159 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:23 smithi027 ceph-mon[30375]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:24.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.158+0000 7f6e20c00640 1 -- 172.21.15.27:0/916701612 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11824 (secure 0 0 0) 0x7f6e140d7cc0 con 0x7f6e241039b0 2024-09-21T01:19:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:24.160 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"btime":"2024-09-21T01:19:21:810458+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24734,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24752,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:19:21.014335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24776,"qdb_cluster":[24776]},"id":1}]} 2024-09-21T01:19:24.160 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-09-21T01:19:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6e080b6560 msgr2=0x7f6e080b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6e080b6560 0x7f6e080b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e24108d70 tx=0x7f6e10005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 msgr2=0x7f6e2419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 0x7f6e2419a2e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e14005ab0 tx=0x7f6e14003c10 comp rx=0 tx=0).stop 2024-09-21T01:19:24.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 shutdown_connections 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6e080b6560 0x7f6e080b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e24109c40 0x7f6e2419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e241092f0 0x7f6e2419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 --2- 172.21.15.27:0/916701612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e241039b0 0x7f6e2419a2e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.161+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 >> 172.21.15.27:0/916701612 conn(0x7f6e240ff400 msgr2=0x7f6e24101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.162+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 shutdown_connections 2024-09-21T01:19:24.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:24.162+0000 7f6e29800640 1 -- 172.21.15.27:0/916701612 wait complete. 2024-09-21T01:19:24.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:23 smithi089 ceph-mon[34486]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:25.055 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:19:25.105 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:24 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/916701612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:19:25.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:24 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/916701612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:19:25.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:24 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/916701612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:19:25.438 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.108+0000 7f4860a00640 1 -- 172.21.15.27:0/1393887414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10b4f0 msgr2=0x7f485c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.108+0000 7f4860a00640 1 --2- 172.21.15.27:0/1393887414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10b4f0 0x7f485c10b900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4844009900 tx=0x7f484402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 -- 172.21.15.27:0/1393887414 shutdown_connections 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 --2- 172.21.15.27:0/1393887414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10be40 0x7f485c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 --2- 172.21.15.27:0/1393887414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10b4f0 0x7f485c10b900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 --2- 172.21.15.27:0/1393887414 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 -- 172.21.15.27:0/1393887414 >> 172.21.15.27:0/1393887414 conn(0x7f485c101680 msgr2=0x7f485c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 -- 172.21.15.27:0/1393887414 shutdown_connections 2024-09-21T01:19:26.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.109+0000 7f4860a00640 1 -- 172.21.15.27:0/1393887414 wait complete. 2024-09-21T01:19:26.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:25 smithi027 ceph-mon[30375]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:26.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.110+0000 7f4860a00640 1 Processor -- start 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.110+0000 7f4860a00640 1 -- start start 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4860a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4860a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 0x7f485c19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4860a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10be40 0x7f485c1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4860a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f485c115b00 con 0x7f485c10be40 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4860a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f485c115980 con 0x7f485c105bb0 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4860a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f485c115c80 con 0x7f485c10b4f0 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4851e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 0x7f485c19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50154/0 (socket says 172.21.15.27:50154) 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 -- 172.21.15.27:0/627614347 learned_addr learned my addr 172.21.15.27:0/627614347 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4851e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 0x7f485c19ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49310/0 (socket says 172.21.15.27:49310) 2024-09-21T01:19:26.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485b200640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10be40 0x7f485c1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 -- 172.21.15.27:0/627614347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 msgr2=0x7f485c19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 0x7f485c19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 -- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10be40 msgr2=0x7f485c1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10be40 0x7f485c1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f485a800640 1 -- 172.21.15.27:0/627614347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c1a2460 con 0x7f485c105bb0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.111+0000 7f4851e00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 0x7f485c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.112+0000 7f485a800640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c19c4f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f484c00e9f0 tx=0x7f484c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.112+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c00cd80 con 0x7f485c105bb0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.112+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c1a2750 con 0x7f485c105bb0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.112+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f484c0108e0 con 0x7f485c105bb0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.112+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c018980 con 0x7f485c105bb0 2024-09-21T01:19:26.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.112+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1a8eb0 con 0x7f485c105bb0 2024-09-21T01:19:26.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.113+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485c105fc0 con 0x7f485c105bb0 2024-09-21T01:19:26.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.116+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f484c010430 con 0x7f485c105bb0 2024-09-21T01:19:26.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.117+0000 7f4853e00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f48400b6560 0x7f48400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.117+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f484c0e7e80 con 0x7f485c105bb0 2024-09-21T01:19:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.117+0000 7f4851e00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f48400b6560 0x7f48400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:26.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.118+0000 7f4851e00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f48400b6560 0x7f48400b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4844009230 tx=0x7f4844031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:26.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.119+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f484c0b2780 con 0x7f485c105bb0 2024-09-21T01:19:26.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:25 smithi098 ceph-mon[34372]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:25 smithi089 ceph-mon[34486]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:26.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.318+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f485c02e640 con 0x7f485c105bb0 2024-09-21T01:19:26.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.320+0000 7f4853e00640 1 -- 172.21.15.27:0/627614347 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24467 (secure 0 0 0) 0x7f484c0b76c0 con 0x7f485c105bb0 2024-09-21T01:19:26.322 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:26.322 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:21.014162+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:19:26.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.322+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f48400b6560 msgr2=0x7f48400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:26.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.322+0000 7f4860a00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f48400b6560 0x7f48400b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4844009230 tx=0x7f4844031400 comp rx=0 tx=0).stop 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.322+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 msgr2=0x7f485c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.322+0000 7f4860a00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c19c4f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f484c00e9f0 tx=0x7f484c00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 shutdown_connections 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f48400b6560 0x7f48400b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f485c10be40 0x7f485c1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f485c10b4f0 0x7f485c19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 --2- 172.21.15.27:0/627614347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f485c105bb0 0x7f485c19c4f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 >> 172.21.15.27:0/627614347 conn(0x7f485c101680 msgr2=0x7f485c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 shutdown_connections 2024-09-21T01:19:26.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:26.323+0000 7f4860a00640 1 -- 172.21.15.27:0/627614347 wait complete. 2024-09-21T01:19:27.057 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:26 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/627614347' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:27.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:19:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:26 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/627614347' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:26 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/627614347' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:27.478 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:27.907 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:27 smithi027 ceph-mon[30375]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:27 smithi098 ceph-mon[34372]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 -- 172.21.15.27:0/1567848557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 msgr2=0x7f0fa4116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 --2- 172.21.15.27:0/1567848557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa4116b60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009ca0 tx=0x7f0f9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 -- 172.21.15.27:0/1567848557 shutdown_connections 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 --2- 172.21.15.27:0/1567848557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa4116b60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 --2- 172.21.15.27:0/1567848557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4078290 0x7f0fa4114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 --2- 172.21.15.27:0/1567848557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fa4077940 0x7f0fa4077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 -- 172.21.15.27:0/1567848557 >> 172.21.15.27:0/1567848557 conn(0x7f0fa4101810 msgr2=0x7f0fa4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:28.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 -- 172.21.15.27:0/1567848557 shutdown_connections 2024-09-21T01:19:28.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.216+0000 7f0fabe00640 1 -- 172.21.15.27:0/1567848557 wait complete. 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.217+0000 7f0fabe00640 1 Processor -- start 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.217+0000 7f0fabe00640 1 -- start start 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.217+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 0x7f0fa41a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fa4078290 0x7f0fa41a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa41a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fabe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa411a0b0 con 0x7f0fa4114740 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fabe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4119f30 con 0x7f0fa4078290 2024-09-21T01:19:28.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fabe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa411a230 con 0x7f0fa4077940 2024-09-21T01:19:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 0x7f0fa41a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 0x7f0fa41a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49316/0 (socket says 172.21.15.27:49316) 2024-09-21T01:19:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 -- 172.21.15.27:0/2715611224 learned_addr learned my addr 172.21.15.27:0/2715611224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9e00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa41a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa8a00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fa4078290 0x7f0fa41a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:28.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 -- 172.21.15.27:0/2715611224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fa4078290 msgr2=0x7f0fa41a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fa4078290 0x7f0fa41a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 -- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 msgr2=0x7f0fa41a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa41a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9400640 1 -- 172.21.15.27:0/2715611224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa41a69f0 con 0x7f0fa4077940 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.218+0000 7f0fa9e00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa41a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.219+0000 7f0fa9400640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 0x7f0fa41a0a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800e9f0 tx=0x7f0f9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.219+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800cd80 con 0x7f0fa4077940 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.219+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa41a6ce0 con 0x7f0fa4077940 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.219+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f980108e0 con 0x7f0fa4077940 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.219+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41ad440 con 0x7f0fa4077940 2024-09-21T01:19:28.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.219+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98018980 con 0x7f0fa4077940 2024-09-21T01:19:28.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.220+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa4077d50 con 0x7f0fa4077940 2024-09-21T01:19:28.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.223+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0f98010430 con 0x7f0fa4077940 2024-09-21T01:19:28.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.225+0000 7f0fa2000640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0f8c0b6560 0x7f0f8c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:28.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.225+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f0f980e7f70 con 0x7f0fa4077940 2024-09-21T01:19:28.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.225+0000 7f0fa8a00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0f8c0b6560 0x7f0f8c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:28.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.226+0000 7f0fa8a00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0f8c0b6560 0x7f0f8c0b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0fa407a1d0 tx=0x7f0f90005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:28.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.226+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f980ae490 con 0x7f0fa4077940 2024-09-21T01:19:28.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:27 smithi089 ceph-mon[34486]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.426+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fa410f930 con 0x7f0fa4077940 2024-09-21T01:19:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.428+0000 7f0fa2000640 1 -- 172.21.15.27:0/2715611224 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24467 (secure 0 0 0) 0x7f0f980b2870 con 0x7f0fa4077940 2024-09-21T01:19:28.430 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:28.430 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:21.014162+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0f8c0b6560 msgr2=0x7f0f8c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0f8c0b6560 0x7f0f8c0b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0fa407a1d0 tx=0x7f0f90005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 msgr2=0x7f0fa41a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 0x7f0fa41a0a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800e9f0 tx=0x7f0f9800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 shutdown_connections 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0f8c0b6560 0x7f0f8c0b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa4114740 0x7f0fa41a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0fa4078290 0x7f0fa41a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 --2- 172.21.15.27:0/2715611224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0fa4077940 0x7f0fa41a0a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 >> 172.21.15.27:0/2715611224 conn(0x7f0fa4101810 msgr2=0x7f0fa4116790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 shutdown_connections 2024-09-21T01:19:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:28.432+0000 7f0fabe00640 1 -- 172.21.15.27:0/2715611224 wait complete. 2024-09-21T01:19:29.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:28 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2715611224' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:29.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:28 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2715611224' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:29.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:19:29.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:28 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2715611224' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:29.619 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:30.086 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:29 smithi027 ceph-mon[30375]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:29 smithi098 ceph-mon[34372]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:29 smithi089 ceph-mon[34486]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:30.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 -- 172.21.15.27:0/4729633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00102460 msgr2=0x7f1b00102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 --2- 172.21.15.27:0/4729633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00102460 0x7f1b00102870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009900 tx=0x7f1af002a480 comp rx=0 tx=0).stop 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 -- 172.21.15.27:0/4729633 shutdown_connections 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 --2- 172.21.15.27:0/4729633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b00110010 0x7f1b00112430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 --2- 172.21.15.27:0/4729633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 --2- 172.21.15.27:0/4729633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00102460 0x7f1b00102870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 -- 172.21.15.27:0/4729633 >> 172.21.15.27:0/4729633 conn(0x7f1b000fe2c0 msgr2=0x7f1b00100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 -- 172.21.15.27:0/4729633 shutdown_connections 2024-09-21T01:19:30.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.267+0000 7f1affe00640 1 -- 172.21.15.27:0/4729633 wait complete. 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.268+0000 7f1affe00640 1 Processor -- start 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 -- start start 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b00102460 0x7f1b0019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00110010 0x7f1b001a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00115ad0 con 0x7f1b00102db0 2024-09-21T01:19:30.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00115950 con 0x7f1b00110010 2024-09-21T01:19:30.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1affe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00115c50 con 0x7f1b00102460 2024-09-21T01:19:30.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1afe000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:30.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1afe000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34188/0 (socket says 172.21.15.27:34188) 2024-09-21T01:19:30.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.269+0000 7f1afe000640 1 -- 172.21.15.27:0/2386335241 learned_addr learned my addr 172.21.15.27:0/2386335241 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:30.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1aff400640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00110010 0x7f1b001a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:30.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afea00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b00102460 0x7f1b0019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afe000640 1 -- 172.21.15.27:0/2386335241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b00102460 msgr2=0x7f1b0019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afe000640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b00102460 0x7f1b0019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afe000640 1 -- 172.21.15.27:0/2386335241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00110010 msgr2=0x7f1b001a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afe000640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00110010 0x7f1b001a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afe000640 1 -- 172.21.15.27:0/2386335241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b001a23f0 con 0x7f1b00102db0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1aff400640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00110010 0x7f1b001a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.270+0000 7f1afe000640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0019ca20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00e9f0 tx=0x7f1aec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.271+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00cd80 con 0x7f1b00102db0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.271+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b001a26e0 con 0x7f1b00102db0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.271+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aec0108e0 con 0x7f1b00102db0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.271+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec018980 con 0x7f1b00102db0 2024-09-21T01:19:30.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.271+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a8e40 con 0x7f1b00102db0 2024-09-21T01:19:30.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.272+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b00102870 con 0x7f1b00102db0 2024-09-21T01:19:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.276+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f1aec010430 con 0x7f1b00102db0 2024-09-21T01:19:30.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.278+0000 7f1af7400640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1ae40b6560 0x7f1ae40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:30.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.279+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f1aec0e7ec0 con 0x7f1b00102db0 2024-09-21T01:19:30.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.279+0000 7f1afea00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1ae40b6560 0x7f1ae40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:30.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.279+0000 7f1afea00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1ae40b6560 0x7f1ae40b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005ab0 tx=0x7f1af0005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:30.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.280+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1aec0b27c0 con 0x7f1b00102db0 2024-09-21T01:19:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.520+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b0019d5b0 con 0x7f1b00102db0 2024-09-21T01:19:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.521+0000 7f1af7400640 1 -- 172.21.15.27:0/2386335241 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11824 (secure 0 0 0) 0x7f1aec0b7700 con 0x7f1b00102db0 2024-09-21T01:19:30.522 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:30.522 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"btime":"2024-09-21T01:19:21:810458+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24674,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24686,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24718,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24734,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24752,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:19:21.014335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24776,"qdb_cluster":[24776]},"id":1}]} 2024-09-21T01:19:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1ae40b6560 msgr2=0x7f1ae40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1ae40b6560 0x7f1ae40b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005ab0 tx=0x7f1af0005a20 comp rx=0 tx=0).stop 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 msgr2=0x7f1b0019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0019ca20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00e9f0 tx=0x7f1aec00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 shutdown_connections 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1ae40b6560 0x7f1ae40b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b00110010 0x7f1b001a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b00102db0 0x7f1b0019ca20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 --2- 172.21.15.27:0/2386335241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b00102460 0x7f1b0019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:30.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 >> 172.21.15.27:0/2386335241 conn(0x7f1b000fe2c0 msgr2=0x7f1b001120d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:30.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 shutdown_connections 2024-09-21T01:19:30.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:30.524+0000 7f1affe00640 1 -- 172.21.15.27:0/2386335241 wait complete. 2024-09-21T01:19:31.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:30 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2386335241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:19:31.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:30 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2386335241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:19:31.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:30 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2386335241' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:19:31.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:19:31.705 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:32.136 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:31 smithi027 ceph-mon[30375]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:32.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:31 smithi098 ceph-mon[34372]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:32.151 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:31 smithi089 ceph-mon[34486]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 -- 172.21.15.27:0/240299495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c90105d80 msgr2=0x7f3c90106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 --2- 172.21.15.27:0/240299495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c90105d80 0x7f3c90106190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009900 tx=0x7f3c8402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 -- 172.21.15.27:0/240299495 shutdown_connections 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 --2- 172.21.15.27:0/240299495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c9010c010 0x7f3c90112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 --2- 172.21.15.27:0/240299495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9010b6c0 0x7f3c9010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 --2- 172.21.15.27:0/240299495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c90105d80 0x7f3c90106190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 -- 172.21.15.27:0/240299495 >> 172.21.15.27:0/240299495 conn(0x7f3c90101810 msgr2=0x7f3c90103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:32.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 -- 172.21.15.27:0/240299495 shutdown_connections 2024-09-21T01:19:32.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.318+0000 7f3c98000640 1 -- 172.21.15.27:0/240299495 wait complete. 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.319+0000 7f3c98000640 1 Processor -- start 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.319+0000 7f3c98000640 1 -- start start 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c98000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 0x7f3c9019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c98000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c9010b6c0 0x7f3c9019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c98000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9010c010 0x7f3c901a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c98000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90115cd0 con 0x7f3c9010b6c0 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c98000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90115b50 con 0x7f3c9010c010 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c98000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c90115e50 con 0x7f3c90105d80 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 0x7f3c9019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:32.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 0x7f3c9019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49376/0 (socket says 172.21.15.27:49376) 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 -- 172.21.15.27:0/1260945392 learned_addr learned my addr 172.21.15.27:0/1260945392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c94c00640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c9010b6c0 0x7f3c9019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c96000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9010c010 0x7f3c901a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 -- 172.21.15.27:0/1260945392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9010c010 msgr2=0x7f3c901a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9010c010 0x7f3c901a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 -- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c9010b6c0 msgr2=0x7f3c9019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c9010b6c0 0x7f3c9019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.320+0000 7f3c95600640 1 -- 172.21.15.27:0/1260945392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c901a2630 con 0x7f3c90105d80 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.321+0000 7f3c95600640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 0x7f3c9019c6c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c84005940 tx=0x7f3c840040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.321+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8403f030 con 0x7f3c90105d80 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.321+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c901a28c0 con 0x7f3c90105d80 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.321+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c840044c0 con 0x7f3c90105d80 2024-09-21T01:19:32.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.321+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84035070 con 0x7f3c90105d80 2024-09-21T01:19:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.321+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c901a9080 con 0x7f3c90105d80 2024-09-21T01:19:32.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.322+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c90106190 con 0x7f3c90105d80 2024-09-21T01:19:32.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.325+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f3c84042030 con 0x7f3c90105d80 2024-09-21T01:19:32.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.326+0000 7f3c8e000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c780b6560 0x7f3c780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:32.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.326+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f3c84107cc0 con 0x7f3c90105d80 2024-09-21T01:19:32.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.327+0000 7f3c94c00640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c780b6560 0x7f3c780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:32.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.327+0000 7f3c94c00640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c780b6560 0x7f3c780b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c9010b1c0 tx=0x7f3c7c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:32.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.328+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c840d25c0 con 0x7f3c90105d80 2024-09-21T01:19:32.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.534+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c9019d7f0 con 0x7f3c90105d80 2024-09-21T01:19:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.535+0000 7f3c8e000640 1 -- 172.21.15.27:0/1260945392 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24467 (secure 0 0 0) 0x7f3c840d7500 con 0x7f3c90105d80 2024-09-21T01:19:32.536 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:32.537 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":86,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:21.014162+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c780b6560 msgr2=0x7f3c780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c780b6560 0x7f3c780b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c9010b1c0 tx=0x7f3c7c005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 msgr2=0x7f3c9019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 0x7f3c9019c6c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c84005940 tx=0x7f3c840040c0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 shutdown_connections 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c780b6560 0x7f3c780b8a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c9010c010 0x7f3c901a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c9010b6c0 0x7f3c9019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 --2- 172.21.15.27:0/1260945392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90105d80 0x7f3c9019c6c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 >> 172.21.15.27:0/1260945392 conn(0x7f3c90101810 msgr2=0x7f3c90103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:32.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 shutdown_connections 2024-09-21T01:19:32.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:32.538+0000 7f3c98000640 1 -- 172.21.15.27:0/1260945392 wait complete. 2024-09-21T01:19:33.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:32 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1260945392' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:33.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:32 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1260945392' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:33.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:32 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1260945392' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:33.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T01:19:07.935504+0000', 'modified': '2024-09-21T01:19:21.014335+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24776': {'gid': 24776, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4156411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4156411896}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4156411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24776, 'qdb_cluster': [24776]} 2024-09-21T01:19:33.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T01:19:33.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T01:19:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:34.140 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:33 smithi027 ceph-mon[30375]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:34.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:33 smithi098 ceph-mon[34372]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:33 smithi089 ceph-mon[34486]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.414+0000 7fc847200640 1 -- 172.21.15.27:0/4207303610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8480757b0 msgr2=0x7fc848075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.414+0000 7fc847200640 1 --2- 172.21.15.27:0/4207303610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8480757b0 0x7fc848075bc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc83802a410 comp rx=0 tx=0).stop 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 -- 172.21.15.27:0/4207303610 shutdown_connections 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 --2- 172.21.15.27:0/4207303610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc848112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 --2- 172.21.15.27:0/4207303610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc848076100 0x7fc84810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 --2- 172.21.15.27:0/4207303610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8480757b0 0x7fc848075bc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 -- 172.21.15.27:0/4207303610 >> 172.21.15.27:0/4207303610 conn(0x7fc8480ff440 msgr2=0x7fc848101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 -- 172.21.15.27:0/4207303610 shutdown_connections 2024-09-21T01:19:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.415+0000 7fc847200640 1 -- 172.21.15.27:0/4207303610 wait complete. 2024-09-21T01:19:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 Processor -- start 2024-09-21T01:19:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 -- start start 2024-09-21T01:19:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8480757b0 0x7fc84819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848076100 0x7fc84819ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc8481a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc848115b10 con 0x7fc848076100 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc848115990 con 0x7fc8480757b0 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.416+0000 7fc847200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc848115c90 con 0x7fc84810ff10 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc8481a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc8481a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49392/0 (socket says 172.21.15.27:49392) 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 -- 172.21.15.27:0/3258031555 learned_addr learned my addr 172.21.15.27:0/3258031555 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc83fe00640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848076100 0x7fc84819ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc845e00640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8480757b0 0x7fc84819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 -- 172.21.15.27:0/3258031555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8480757b0 msgr2=0x7fc84819c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8480757b0 0x7fc84819c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 -- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848076100 msgr2=0x7fc84819ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848076100 0x7fc84819ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc846800640 1 -- 172.21.15.27:0/3258031555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a23f0 con 0x7fc84810ff10 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.417+0000 7fc83fe00640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848076100 0x7fc84819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.418+0000 7fc846800640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc8481a1cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc84000eb10 tx=0x7fc84000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.418+0000 7fc83ea00640 1 -- 172.21.15.27:0/3258031555 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004b20 con 0x7fc84810ff10 2024-09-21T01:19:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.418+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a26e0 con 0x7fc84810ff10 2024-09-21T01:19:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.418+0000 7fc83ea00640 1 -- 172.21.15.27:0/3258031555 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc840004cc0 con 0x7fc84810ff10 2024-09-21T01:19:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.418+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a8e40 con 0x7fc84810ff10 2024-09-21T01:19:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.418+0000 7fc83ea00640 1 -- 172.21.15.27:0/3258031555 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840010690 con 0x7fc84810ff10 2024-09-21T01:19:34.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.422+0000 7fc85385f740 1 Processor -- start 2024-09-21T01:19:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.423+0000 7fc83ea00640 1 -- 172.21.15.27:0/3258031555 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fc840005000 con 0x7fc84810ff10 2024-09-21T01:19:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.423+0000 7fc85385f740 1 -- start start 2024-09-21T01:19:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc85385f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 0x556182bcc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc85385f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561827fd0a0 0x556182bccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc85385f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bcd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc85385f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556182b43d50 con 0x556182805280 2024-09-21T01:19:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc85385f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556182b33180 con 0x5561827fd0a0 2024-09-21T01:19:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc85385f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556182b36f90 con 0x556182825f60 2024-09-21T01:19:34.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc83fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bcd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc845e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561827fd0a0 0x556182bccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc83fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 0x556182bcc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc83fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 0x556182bcc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49408/0 (socket says 172.21.15.27:49408) 2024-09-21T01:19:34.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc845e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561827fd0a0 0x556182bccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:50234/0 (socket says 172.21.15.27:50234) 2024-09-21T01:19:34.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.424+0000 7fc83fe00640 1 -- 172.21.15.27:0/142021563 learned_addr learned my addr 172.21.15.27:0/142021563 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83fe00640 1 -- 172.21.15.27:0/142021563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561827fd0a0 msgr2=0x556182bccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83fe00640 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561827fd0a0 0x556182bccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83fe00640 1 -- 172.21.15.27:0/142021563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 msgr2=0x556182bcd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83fe00640 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bcd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83fe00640 1 -- 172.21.15.27:0/142021563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556182afda70 con 0x556182825f60 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83ea00640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc82c0b6560 0x7fc82c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83fe00640 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 0x556182bcc670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8300127d0 tx=0x7fc830005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bbeb7291ce509a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc83ea00640 1 -- 172.21.15.27:0/3258031555 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fc8400e7cf0 con 0x7fc84810ff10 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.425+0000 7fc845e00640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc82c0b6560 0x7fc82c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.426+0000 7fc837000640 1 -- 172.21.15.27:0/142021563 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830012110 con 0x556182825f60 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.426+0000 7fc837000640 1 -- 172.21.15.27:0/142021563 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83000fc70 con 0x556182825f60 2024-09-21T01:19:34.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.426+0000 7fc837000640 1 -- 172.21.15.27:0/142021563 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830010a20 con 0x556182825f60 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 -- 172.21.15.27:0/142021563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 msgr2=0x556182bcc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 0x556182bcc670 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8300127d0 tx=0x7fc830005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc845e00640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc82c0b6560 0x7fc82c0b8a70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc838005ab0 tx=0x7fc838005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 -- 172.21.15.27:0/142021563 shutdown_connections 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bcd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5561827fd0a0 0x556182bccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 --2- 172.21.15.27:0/142021563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556182825f60 0x556182bcc670 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 -- 172.21.15.27:0/142021563 >> 172.21.15.27:0/142021563 conn(0x55618281a350 msgr2=0x556182bca560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 -- 172.21.15.27:0/142021563 shutdown_connections 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 -- 172.21.15.27:0/142021563 wait complete. 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.427+0000 7fc85385f740 1 -- start start 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 Processor -- start 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bd8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182805280 0x556182bd86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182825f60 0x556182be4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5561829945f0 con 0x556182825f60 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556182ace460 con 0x556182805280 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556182bdae50 con 0x5561827fd0a0 2024-09-21T01:19:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.428+0000 7fc85385f740 10 client.0 _ll_register_callbacks cb 0x556182ade210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc846800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182825f60 0x556182be4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc845e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bd8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182805280 0x556182bd86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc845e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bd8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49424/0 (socket says 172.21.15.27:49424) 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc846800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182825f60 0x556182be4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34230/0 (socket says 172.21.15.27:34230) 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc845e00640 1 -- 172.21.15.27:0/2343399869 learned_addr learned my addr 172.21.15.27:0/2343399869 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 -- 172.21.15.27:0/2343399869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 msgr2=0x556182bd8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bd8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 -- 172.21.15.27:0/2343399869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182825f60 msgr2=0x556182be4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182825f60 0x556182be4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 -- 172.21.15.27:0/2343399869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556182891540 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc83fe00640 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182805280 0x556182bd86a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc83000a010 tx=0x7fc8300050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70486e08053fa1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.429+0000 7fc82b400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc82b400640 1 -- 172.21.15.27:0/2343399869 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830023020 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc82b400640 1 -- 172.21.15.27:0/2343399869 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fc830004050 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556182ad0a70 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc82b400640 1 -- 172.21.15.27:0/2343399869 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc83002e050 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc85385f740 10 client.24872 fetch_fsmap learned FSMap version 17 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556182aeb6f0 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.430+0000 7fc82b400640 1 -- 172.21.15.27:0/2343399869 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556182ad0a70 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc82b400640 1 -- 172.21.15.27:0/2343399869 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fc83000f7c0 con 0x556182805280 2024-09-21T01:19:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc85385f740 10 client.24872 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T01:19:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc85385f740 10 client.24872 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:19:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc85385f740 20 client.24872 populate_metadata read hostname 'smithi027' 2024-09-21T01:19:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc85385f740 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827e48e0 0x556182bea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc85385f740 4 client.24872 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:19:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556182b45ca0 con 0x5561827e48e0 2024-09-21T01:19:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.431+0000 7fc845e00640 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827e48e0 0x556182bea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.432+0000 7fc845e00640 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827e48e0 0x556182bea3c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff8f35f2281a126 server_cookie=f9ed9f0657141de1 in_seq=0 out_seq=0 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.432+0000 7fc82b400640 10 client.24872 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc82b400640 1 -- 172.21.15.27:0/2343399869 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556182b45ca0 con 0x5561827e48e0 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc82b400640 10 client.24872 handle_command_reply: tid=0 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 client.24872 shutdown 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182805280 msgr2=0x556182bd86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182805280 0x556182bd86a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc83000a010 tx=0x7fc8300050c0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 shutdown_connections 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182825f60 0x556182be4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827e48e0 0x556182bea3c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182805280 0x556182bd86a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 --2- 172.21.15.27:0/2343399869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bd8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc82b400640 0 client.24872 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.433+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 >> 172.21.15.27:0/2343399869 conn(0x55618281a350 msgr2=0x556182bcf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.434+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 shutdown_connections 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.434+0000 7fc85385f740 1 -- 172.21.15.27:0/2343399869 wait complete. 2024-09-21T01:19:34.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.434+0000 7fc85385f740 20 client.24872 trim_cache size 0 max 16384 2024-09-21T01:19:34.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 Processor -- start 2024-09-21T01:19:34.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 -- start start 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182be9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182be40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182be4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556182be8700 con 0x556182805280 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556182be8dd0 con 0x556182825f60 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.453+0000 7fc85385f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556182be94a0 con 0x5561827fd0a0 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182be9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182be9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49440/0 (socket says 172.21.15.27:49440) 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 -- 172.21.15.27:0/3824083598 learned_addr learned my addr 172.21.15.27:0/3824083598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc83fe00640 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182be40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc846800640 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182be4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 -- 172.21.15.27:0/3824083598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 msgr2=0x556182be4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182be4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 -- 172.21.15.27:0/3824083598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 msgr2=0x556182be40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182be40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 -- 172.21.15.27:0/3824083598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556182bcf4d0 con 0x5561827fd0a0 2024-09-21T01:19:34.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.454+0000 7fc845e00640 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182be9620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc83802aa00 tx=0x7fc83803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0bc4198ac27a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.455+0000 7fc836600640 1 -- 172.21.15.27:0/3824083598 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838035040 con 0x5561827fd0a0 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.455+0000 7fc836600640 1 -- 172.21.15.27:0/3824083598 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc838004e20 con 0x5561827fd0a0 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.455+0000 7fc836600640 1 -- 172.21.15.27:0/3824083598 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380324b0 con 0x5561827fd0a0 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.455+0000 7fc85385f740 1 -- 172.21.15.27:0/3824083598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 msgr2=0x556182be9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.455+0000 7fc85385f740 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182be9620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc83802aa00 tx=0x7fc83803f000 comp rx=0 tx=0).stop 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 -- 172.21.15.27:0/3824083598 shutdown_connections 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182be4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182be40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 --2- 172.21.15.27:0/3824083598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182be9620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 -- 172.21.15.27:0/3824083598 >> 172.21.15.27:0/3824083598 conn(0x55618281a350 msgr2=0x556182bd3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:34.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 -- 172.21.15.27:0/3824083598 shutdown_connections 2024-09-21T01:19:34.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 -- 172.21.15.27:0/3824083598 wait complete. 2024-09-21T01:19:34.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 -- start start 2024-09-21T01:19:34.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.456+0000 7fc85385f740 1 Processor -- start 2024-09-21T01:19:34.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bf0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bf0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182bebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556182bea240 con 0x556182805280 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556182be9b70 con 0x556182825f60 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556182bea910 con 0x5561827fd0a0 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc85385f740 10 client.0 _ll_register_callbacks cb 0x7fc8300126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc83fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bf0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc845e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bf0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc83fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bf0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34238/0 (socket says 172.21.15.27:34238) 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc83fe00640 1 -- 172.21.15.27:0/4081756433 learned_addr learned my addr 172.21.15.27:0/4081756433 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.457+0000 7fc846800640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182bebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc83fe00640 1 -- 172.21.15.27:0/4081756433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 msgr2=0x556182bf0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc83fe00640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bf0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc83fe00640 1 -- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 msgr2=0x556182bebcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc83fe00640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182bebcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc83fe00640 1 -- 172.21.15.27:0/4081756433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556182bec3b0 con 0x556182805280 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc846800640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182bebcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc83fe00640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bf0c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc830004600 tx=0x7fc83000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cece1ea441462c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc82b400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.458+0000 7fc82b400640 1 -- 172.21.15.27:0/4081756433 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830023040 con 0x556182805280 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc82b400640 1 -- 172.21.15.27:0/4081756433 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fc83000f790 con 0x556182805280 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc838004e20 con 0x556182805280 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc82b400640 1 -- 172.21.15.27:0/4081756433 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc83003f050 con 0x556182805280 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc85385f740 10 client.14832 fetch_fsmap learned FSMap version 17 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556182bf1de0 con 0x556182805280 2024-09-21T01:19:34.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc82b400640 1 -- 172.21.15.27:0/4081756433 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc838004e20 con 0x556182805280 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc82b400640 1 -- 172.21.15.27:0/4081756433 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fc83003a030 con 0x556182805280 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc85385f740 10 client.14832 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.459+0000 7fc85385f740 10 client.14832 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.460+0000 7fc85385f740 20 client.14832 populate_metadata read hostname 'smithi027' 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.460+0000 7fc85385f740 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827b9c90 0x556182c58b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.460+0000 7fc85385f740 4 client.14832 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.460+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "status"}) -- 0x556182b41750 con 0x5561827b9c90 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.460+0000 7fc845e00640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827b9c90 0x556182c58b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:34.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.460+0000 7fc845e00640 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827b9c90 0x556182c58b70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec72d44f1be183a server_cookie=2dcfa7dc80ed0d87 in_seq=0 out_seq=0 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc82b400640 10 client.14832 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc82b400640 1 -- 172.21.15.27:0/4081756433 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556182b41750 con 0x5561827b9c90 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc82b400640 10 client.14832 handle_command_reply: tid=0 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 client.14832 shutdown 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 msgr2=0x556182bf0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bf0c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc830004600 tx=0x7fc83000ff60 comp rx=0 tx=0).stop 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 shutdown_connections 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5561827b9c90 0x556182c58b70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556182825f60 0x556182bebcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556182805280 0x556182bf0c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 --2- 172.21.15.27:0/4081756433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5561827fd0a0 0x556182bf0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 >> 172.21.15.27:0/4081756433 conn(0x55618281a350 msgr2=0x556182bd66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:34.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc82b400640 0 client.14832 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 shutdown_connections 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.461+0000 7fc85385f740 1 -- 172.21.15.27:0/4081756433 wait complete. 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.462+0000 7fc85385f740 20 client.14832 trim_cache size 0 max 16384 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24776, 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:19:34.465 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 25.731105481, 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 86, 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 40.239044913999997 2024-09-21T01:19:34.466 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.462+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc82c0b6560 msgr2=0x7fc82c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.462+0000 7fc847200640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc82c0b6560 0x7fc82c0b8a70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc838005ab0 tx=0x7fc838005a20 comp rx=0 tx=0).stop 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.462+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 msgr2=0x7fc8481a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.462+0000 7fc847200640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc8481a1cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc84000eb10 tx=0x7fc84000c2d0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 shutdown_connections 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc82c0b6560 0x7fc82c0b8a70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc84810ff10 0x7fc8481a1cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc848076100 0x7fc84819ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 --2- 172.21.15.27:0/3258031555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8480757b0 0x7fc84819c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:34.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 >> 172.21.15.27:0/3258031555 conn(0x7fc8480ff440 msgr2=0x7fc848109a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:34.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 shutdown_connections 2024-09-21T01:19:34.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:34.463+0000 7fc847200640 1 -- 172.21.15.27:0/3258031555 wait complete. 2024-09-21T01:19:35.392 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T01:19:35.392 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:35.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T01:19:35.782 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:36.126 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:35 smithi027 ceph-mon[30375]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:36.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:35 smithi098 ceph-mon[34372]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:36.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:35 smithi089 ceph-mon[34486]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:36.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.449+0000 7f8972a00640 1 -- 172.21.15.27:0/1210887282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 msgr2=0x7f896c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.449+0000 7f8972a00640 1 --2- 172.21.15.27:0/1210887282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c1098b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8958009900 tx=0x7f895802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 -- 172.21.15.27:0/1210887282 shutdown_connections 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 --2- 172.21.15.27:0/1210887282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896c109df0 0x7f896c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 --2- 172.21.15.27:0/1210887282 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c1098b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 --2- 172.21.15.27:0/1210887282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f896c103b60 0x7f896c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 -- 172.21.15.27:0/1210887282 >> 172.21.15.27:0/1210887282 conn(0x7f896c07b850 msgr2=0x7f896c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 -- 172.21.15.27:0/1210887282 shutdown_connections 2024-09-21T01:19:36.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.450+0000 7f8972a00640 1 -- 172.21.15.27:0/1210887282 wait complete. 2024-09-21T01:19:36.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 Processor -- start 2024-09-21T01:19:36.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 -- start start 2024-09-21T01:19:36.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f896c103b60 0x7f896c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:36.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896c109df0 0x7f896c1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f896c113ab0 con 0x7f896c109df0 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f896c113930 con 0x7f896c1094a0 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.451+0000 7f8972a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f896c113c30 con 0x7f896c103b60 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f8970a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896c109df0 0x7f896c1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53752/0 (socket says 172.21.15.27:53752) 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 -- 172.21.15.27:0/170620685 learned_addr learned my addr 172.21.15.27:0/170620685 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:36.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896be00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f896c103b60 0x7f896c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 -- 172.21.15.27:0/170620685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f896c103b60 msgr2=0x7f896c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f896c103b60 0x7f896c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 -- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896c109df0 msgr2=0x7f896c1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896c109df0 0x7f896c1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.452+0000 7f896b400640 1 -- 172.21.15.27:0/170620685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c1a25f0 con 0x7f896c1094a0 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.453+0000 7f896b400640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c19cc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f895802a910 tx=0x7f8958031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:36.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.453+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895803f030 con 0x7f896c1094a0 2024-09-21T01:19:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.453+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8958004490 con 0x7f896c1094a0 2024-09-21T01:19:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.453+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c1a2880 con 0x7f896c1094a0 2024-09-21T01:19:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.453+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958035070 con 0x7f896c1094a0 2024-09-21T01:19:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.453+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1a9040 con 0x7f896c1094a0 2024-09-21T01:19:36.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.454+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896c104de0 con 0x7f896c1094a0 2024-09-21T01:19:36.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.457+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f8958042030 con 0x7f896c1094a0 2024-09-21T01:19:36.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.458+0000 7f8968c00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f89500b6560 0x7f89500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:36.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.458+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f8958107db0 con 0x7f896c1094a0 2024-09-21T01:19:36.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.458+0000 7f896be00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f89500b6560 0x7f89500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:36.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.459+0000 7f896be00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f89500b6560 0x7f89500b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f895c0095f0 tx=0x7f895c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.460+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89580d26b0 con 0x7f896c1094a0 2024-09-21T01:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.701+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f896c103f70 con 0x7f896c1094a0 2024-09-21T01:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.702+0000 7f8968c00640 1 -- 172.21.15.27:0/170620685 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f896c103f70 con 0x7f896c1094a0 2024-09-21T01:19:36.703 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T01:19:36.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f89500b6560 msgr2=0x7f89500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:36.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f89500b6560 0x7f89500b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f895c0095f0 tx=0x7f895c005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:36.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 msgr2=0x7f896c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:36.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c19cc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f895802a910 tx=0x7f8958031a20 comp rx=0 tx=0).stop 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 shutdown_connections 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f89500b6560 0x7f89500b8a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896c109df0 0x7f896c1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f896c1094a0 0x7f896c19cc20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 --2- 172.21.15.27:0/170620685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f896c103b60 0x7f896c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.704+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 >> 172.21.15.27:0/170620685 conn(0x7f896c07b850 msgr2=0x7f896c107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.705+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 shutdown_connections 2024-09-21T01:19:36.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:36.705+0000 7f8972a00640 1 -- 172.21.15.27:0/170620685 wait complete. 2024-09-21T01:19:37.104 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:36 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/170620685' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:19:37.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:36 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/170620685' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:19:37.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:36 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/170620685' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:19:37.476 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T01:19:37.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T01:19:37.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-21T01:19:37.476 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T01:19:37.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:37.476 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T01:19:37.477 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T01:19:37.477 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T01:19:37.477 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T01:19:37.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:37.477 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-21T01:19:37.514 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T01:19:37.514 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-21T01:19:37.515 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-21T01:19:37.516 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T01:19:37.517 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42109sec preferred_lft 42109sec 2024-09-21T01:19:37.517 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T01:19:37.517 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:19:37.517 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:19:37.517 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T01:19:37.517 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-21T01:19:37.518 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T01:19:37.518 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:19:37.518 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:19:37.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-21T01:19:37.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-21T01:19:37.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-21T01:19:37.519 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T01:19:37.519 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:19:37.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:19:37.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:37.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:37.964 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T01:19:38.026 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-21T01:19:38.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:38.029 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-21T01:19:38.081 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-21T01:19:38.081 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T01:19:38.081 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T01:19:38.081 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T01:19:38.081 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T01:19:38.082 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:19:38.082 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:19:38.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T01:19:38.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T01:19:38.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T01:19:38.082 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:19:38.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:37 smithi098 ceph-mon[34372]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:19:38.179 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:37 smithi027 ceph-mon[30375]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:38.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:37 smithi089 ceph-mon[34486]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:38.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:38.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:38.722 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T01:19:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:38.759 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-21T01:19:38.836 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:19:38.836 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:19:38.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:38.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T01:19:38.837 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:19:38.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:19:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:39.021 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T01:19:39.021 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:19:39.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:19:39.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:39.263 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:19:39.263 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:19:39.264 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-21T01:19:39.264 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T01:19:39.264 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:19:39.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:19:39.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:39.433 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T01:19:39.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T01:19:39.433 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:39.469 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T01:19:39.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T01:19:39.470 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:39.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T01:19:39.644 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T01:19:39.681 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T01:19:39.681 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T01:19:39.681 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T01:19:39.681 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T01:19:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/4b4efff80ec1e461475ab6be8b73394a55e47056e58326069a4f742b8d1d4a1c/merged 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bf1b40b68a78f31f017f4696c4a764eae9bde223e45bb402ae4c92ec6f179a7f/merged 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T01:19:39.683 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/c976d645fb4624ad2cd5ef504013ac3a614d33c92093ca1d8c15e1913c8d80ae/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6731d14e1533e8048be7ee5ddee65634b701e48ec19f9f8fafbcb487669a98d6/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/030956908a0b17b8174ff603760cbea8569a20a05fc6b2e976437c46a2acf468/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/ab84b976d9d9d5aff02c18d8a00a02763c5b7c337c4c4a666c373438c1868398/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/36097a442394974f833b0f423bfe3906e100a2c5c4066ac4a6e1f996f66728d7/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/dc0edcba87abd7152a68237f23672fa83f10268462cbaf7777d2baf5d8b1814f/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/980c76862b9dec71986ed15c6042185379db5e54e8e34d029763564e35334f8e/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/aad1102280f41456992acfbeebcbb8a1ac50ba0ef5dccd8a5a6d80c41f296c8d/merged 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:39.684 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:39.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:39.685 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T01:19:39.761 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T01:19:39.762 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T01:19:39.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T01:19:39.863 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T01:19:39.900 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:39 smithi027 ceph-mon[30375]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:39.901 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T01:19:39.901 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T01:19:39.901 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T01:19:39.901 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T01:19:39.901 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T01:19:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/4b4efff80ec1e461475ab6be8b73394a55e47056e58326069a4f742b8d1d4a1c/merged 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bf1b40b68a78f31f017f4696c4a764eae9bde223e45bb402ae4c92ec6f179a7f/merged 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/c976d645fb4624ad2cd5ef504013ac3a614d33c92093ca1d8c15e1913c8d80ae/merged 2024-09-21T01:19:39.903 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6731d14e1533e8048be7ee5ddee65634b701e48ec19f9f8fafbcb487669a98d6/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/030956908a0b17b8174ff603760cbea8569a20a05fc6b2e976437c46a2acf468/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/ab84b976d9d9d5aff02c18d8a00a02763c5b7c337c4c4a666c373438c1868398/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/36097a442394974f833b0f423bfe3906e100a2c5c4066ac4a6e1f996f66728d7/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/dc0edcba87abd7152a68237f23672fa83f10268462cbaf7777d2baf5d8b1814f/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/980c76862b9dec71986ed15c6042185379db5e54e8e34d029763564e35334f8e/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/aad1102280f41456992acfbeebcbb8a1ac50ba0ef5dccd8a5a6d80c41f296c8d/merged 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:39.904 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:39.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:39.905 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T01:19:40.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:2024-09-21T01:19:40.065+0000 7f7dcfd40480 -1 init, newargv = 0x7f7db4054370 newargc=15 2024-09-21T01:19:40.066 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[76489]: starting ceph client 2024-09-21T01:19:40.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[76489]: starting fuse 2024-09-21T01:19:40.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:39 smithi098 ceph-mon[34372]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:40.252 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:39 smithi089 ceph-mon[34486]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:19:41.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2024-09-21T01:19:41.137 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9, 5, 7]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]} : dispatch 2024-09-21T01:19:41.138 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:40 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]} : dispatch 2024-09-21T01:19:41.145 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T01:19:41.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9, 5, 7]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]} : dispatch 2024-09-21T01:19:41.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:40 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]} : dispatch 2024-09-21T01:19:41.182 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2024-09-21T01:19:41.183 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/4b4efff80ec1e461475ab6be8b73394a55e47056e58326069a4f742b8d1d4a1c/merged 2024-09-21T01:19:41.184 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/bf1b40b68a78f31f017f4696c4a764eae9bde223e45bb402ae4c92ec6f179a7f/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/0 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/c976d645fb4624ad2cd5ef504013ac3a614d33c92093ca1d8c15e1913c8d80ae/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/6731d14e1533e8048be7ee5ddee65634b701e48ec19f9f8fafbcb487669a98d6/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/030956908a0b17b8174ff603760cbea8569a20a05fc6b2e976437c46a2acf468/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/ab84b976d9d9d5aff02c18d8a00a02763c5b7c337c4c4a666c373438c1868398/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/36097a442394974f833b0f423bfe3906e100a2c5c4066ac4a6e1f996f66728d7/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/dc0edcba87abd7152a68237f23672fa83f10268462cbaf7777d2baf5d8b1814f/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/980c76862b9dec71986ed15c6042185379db5e54e8e34d029763564e35334f8e/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/containers/storage/overlay/aad1102280f41456992acfbeebcbb8a1ac50ba0ef5dccd8a5a6d80c41f296c8d/merged 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:19:41.185 INFO:teuthology.orchestra.run.smithi027.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:41.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:41.186 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2024-09-21T01:19:41.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9, 5, 7]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]} : dispatch 2024-09-21T01:19:41.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:40 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]} : dispatch 2024-09-21T01:19:41.263 INFO:teuthology.orchestra.run.smithi027.stdout:120 2024-09-21T01:19:41.263 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T01:19:41.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> import re 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> import os 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2024-09-21T01:19:41.264 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> return f 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.admin")) 2024-09-21T01:19:41.265 DEBUG:teuthology.orchestra.run.smithi027:> ' 2024-09-21T01:19:41.430 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.admin.76489.asok 2024-09-21T01:19:41.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:41.445 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76489.asok 2024-09-21T01:19:41.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:41.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76489.asok status 2024-09-21T01:19:41.701 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "admin", 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "76489", 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2024-09-21T01:19:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 14844, 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 14844 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1639575298 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2024-09-21T01:19:41.703 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1639575298 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.14844 192.168.0.1:0/1639575298", 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/1639575298", 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 17, 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 86, 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2024-09-21T01:19:41.704 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:19:41.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:41.720 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2024-09-21T01:19:41.720 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:41.756 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2024-09-21T01:19:41.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:41.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:41.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:41.855 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T01:19:41.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:19:41.973 INFO:teuthology.orchestra.run.smithi027.stdout:# file: . 2024-09-21T01:19:41.973 INFO:teuthology.orchestra.run.smithi027.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T01:19:41.973 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:19:41.981 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:19:41.981 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T01:19:42.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T01:19:42.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T01:19:42.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9, 5, 7]}]': finished 2024-09-21T01:19:42.111 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-21T01:19:42.112 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:41 smithi027 ceph-mon[30375]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9, 5, 7]}]': finished 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-21T01:19:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-21T01:19:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-21T01:19:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-21T01:19:42.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:41 smithi098 ceph-mon[34372]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T01:19:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-21T01:19:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9, 5, 7]}]': finished 2024-09-21T01:19:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T01:19:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-21T01:19:42.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:41 smithi089 ceph-mon[34486]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T01:19:42.447 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:43.148 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:42 smithi027 ceph-mon[30375]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T01:19:43.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 -- 172.21.15.27:0/2204875675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 msgr2=0x7fc350105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:43.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 --2- 172.21.15.27:0/2204875675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc350105fc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc344009900 tx=0x7fc34402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 -- 172.21.15.27:0/2204875675 shutdown_connections 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 --2- 172.21.15.27:0/2204875675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 0x7fc350112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 --2- 172.21.15.27:0/2204875675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 0x7fc35010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 --2- 172.21.15.27:0/2204875675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc350105fc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 -- 172.21.15.27:0/2204875675 >> 172.21.15.27:0/2204875675 conn(0x7fc350101680 msgr2=0x7fc350103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 -- 172.21.15.27:0/2204875675 shutdown_connections 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.147+0000 7fc356000640 1 -- 172.21.15.27:0/2204875675 wait complete. 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.149+0000 7fc356000640 1 Processor -- start 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.149+0000 7fc356000640 1 -- start start 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.149+0000 7fc356000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc35019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc356000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 0x7fc35019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc356000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 0x7fc3501a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc356000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc350115b00 con 0x7fc350105bb0 2024-09-21T01:19:43.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc356000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc350115980 con 0x7fc35010b4f0 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc356000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc350115c80 con 0x7fc35010be40 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc35019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc35019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48352/0 (socket says 172.21.15.27:48352) 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 -- 172.21.15.27:0/3310454289 learned_addr learned my addr 172.21.15.27:0/3310454289 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc355600640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 0x7fc3501a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc34fe00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 0x7fc35019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 -- 172.21.15.27:0/3310454289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 msgr2=0x7fc3501a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 0x7fc3501a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 -- 172.21.15.27:0/3310454289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 msgr2=0x7fc35019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.150+0000 7fc354c00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 0x7fc35019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc354c00640 1 -- 172.21.15.27:0/3310454289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3501a2480 con 0x7fc350105bb0 2024-09-21T01:19:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc355600640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 0x7fc3501a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc34fe00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 0x7fc35019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc354c00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc35019c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc344005940 tx=0x7fc3440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34403f030 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3440044c0 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3501a2710 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344035070 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.151+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501a8ed0 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.152+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc350106e30 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.158+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fc344042030 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.159+0000 7fc34d600640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc3380b6560 0x7fc3380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.160+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fc344107d50 con 0x7fc350105bb0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.160+0000 7fc34fe00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc3380b6560 0x7fc3380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.160+0000 7fc34fe00640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc3380b6560 0x7fc3380b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0095f0 tx=0x7fc33c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.162+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3440d2540 con 0x7fc350105bb0 2024-09-21T01:19:43.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:42 smithi089 ceph-mon[34486]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T01:19:43.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.374+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc350105fc0 con 0x7fc350105bb0 2024-09-21T01:19:43.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.375+0000 7fc34d600640 1 -- 172.21.15.27:0/3310454289 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v88) ==== 81+0+885 (secure 0 0 0) 0x7fc3440d7480 con 0x7fc350105bb0 2024-09-21T01:19:43.375 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/525441395 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3673025816 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3673025816 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3489401183 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2219512661 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1438694632 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3049262375 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3134270458 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3049262375 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/365392204 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1444171027 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/492430082 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1523735126 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:43.376 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1528176571 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:43.377 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1523735126 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:43.377 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc3380b6560 msgr2=0x7fc3380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc3380b6560 0x7fc3380b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0095f0 tx=0x7fc33c005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 msgr2=0x7fc35019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc35019c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc344005940 tx=0x7fc3440040c0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 shutdown_connections 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc3380b6560 0x7fc3380b8a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc35010be40 0x7fc3501a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc35010b4f0 0x7fc35019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 --2- 172.21.15.27:0/3310454289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc350105bb0 0x7fc35019c510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:43.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.377+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 >> 172.21.15.27:0/3310454289 conn(0x7fc350101680 msgr2=0x7fc350103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:43.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.378+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 shutdown_connections 2024-09-21T01:19:43.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:43.378+0000 7fc356000640 1 -- 172.21.15.27:0/3310454289 wait complete. 2024-09-21T01:19:43.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:42 smithi098 ceph-mon[34372]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T01:19:44.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:43 smithi027 ceph-mon[30375]: pgmap v317: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:44.178 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:43 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3310454289' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:19:44.213 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:19:44.213 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T01:19:44.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T01:19:44.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:43 smithi089 ceph-mon[34486]: pgmap v317: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:44.251 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:43 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3310454289' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:19:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:43 smithi098 ceph-mon[34372]: pgmap v317: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:43 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3310454289' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:19:44.677 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 -- 172.21.15.27:0/224353964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c1049b0 msgr2=0x7fab3c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 --2- 172.21.15.27:0/224353964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c1049b0 0x7fab3c104dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009900 tx=0x7fab2c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 -- 172.21.15.27:0/224353964 shutdown_connections 2024-09-21T01:19:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 --2- 172.21.15.27:0/224353964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab3c10de10 0x7fab3c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 --2- 172.21.15.27:0/224353964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c105300 0x7fab3c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 --2- 172.21.15.27:0/224353964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c1049b0 0x7fab3c104dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 -- 172.21.15.27:0/224353964 >> 172.21.15.27:0/224353964 conn(0x7fab3c079720 msgr2=0x7fab3c079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:45.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 -- 172.21.15.27:0/224353964 shutdown_connections 2024-09-21T01:19:45.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.337+0000 7fab43000640 1 -- 172.21.15.27:0/224353964 wait complete. 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.338+0000 7fab43000640 1 Processor -- start 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.338+0000 7fab43000640 1 -- start start 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.338+0000 7fab43000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 0x7fab3c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab43000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c105300 0x7fab3c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab43000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab3c10de10 0x7fab3c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab43000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c113ae0 con 0x7fab3c105300 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab43000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c113960 con 0x7fab3c1049b0 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab43000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c113c60 con 0x7fab3c10de10 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 0x7fab3c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:45.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 0x7fab3c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53804/0 (socket says 172.21.15.27:53804) 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 -- 172.21.15.27:0/2456738363 learned_addr learned my addr 172.21.15.27:0/2456738363 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab41000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab3c10de10 0x7fab3c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab33400640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c105300 0x7fab3c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 -- 172.21.15.27:0/2456738363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab3c10de10 msgr2=0x7fab3c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab3c10de10 0x7fab3c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 -- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c105300 msgr2=0x7fab3c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c105300 0x7fab3c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab3be00640 1 -- 172.21.15.27:0/2456738363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c1a0400 con 0x7fab3c1049b0 2024-09-21T01:19:45.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.339+0000 7fab33400640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c105300 0x7fab3c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.340+0000 7fab3be00640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 0x7fab3c19a490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005ab0 tx=0x7fab2c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.340+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c03f070 con 0x7fab3c1049b0 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.340+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c1a0690 con 0x7fab3c1049b0 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.340+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c031d60 con 0x7fab3c1049b0 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.340+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c035030 con 0x7fab3c1049b0 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.340+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c1025d0 con 0x7fab3c1049b0 2024-09-21T01:19:45.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.341+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab3c06c2e0 con 0x7fab3c1049b0 2024-09-21T01:19:45.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.344+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fab2c042070 con 0x7fab3c1049b0 2024-09-21T01:19:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.345+0000 7fab39600640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab200b65e0 0x7fab200b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.345+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fab2c108680 con 0x7fab3c1049b0 2024-09-21T01:19:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.346+0000 7fab33400640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab200b65e0 0x7fab200b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:45.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.346+0000 7fab33400640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab200b65e0 0x7fab200b8af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab24005d00 tx=0x7fab24005c70 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:45.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.347+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab2c0cea90 con 0x7fab3c1049b0 2024-09-21T01:19:45.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.560+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fab3c104dc0 con 0x7fab3c1049b0 2024-09-21T01:19:45.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.560+0000 7fab39600640 1 -- 172.21.15.27:0/2456738363 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v88) ==== 81+0+885 (secure 0 0 0) 0x7fab2c0d2e70 con 0x7fab3c1049b0 2024-09-21T01:19:45.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/525441395 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:45.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3673025816 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:45.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3673025816 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:45.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3489401183 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:45.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2219512661 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1438694632 2024-09-22T01:10:00.491967+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3049262375 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3134270458 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3049262375 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/365392204 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1444171027 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/492430082 2024-09-22T01:10:32.417709+0000 2024-09-21T01:19:45.564 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1523735126 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:45.564 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1528176571 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:45.564 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1523735126 2024-09-22T01:11:40.306879+0000 2024-09-21T01:19:45.565 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T01:19:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab200b65e0 msgr2=0x7fab200b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab200b65e0 0x7fab200b8af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab24005d00 tx=0x7fab24005c70 comp rx=0 tx=0).stop 2024-09-21T01:19:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 msgr2=0x7fab3c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 0x7fab3c19a490 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fab2c005ab0 tx=0x7fab2c003cd0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 shutdown_connections 2024-09-21T01:19:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fab200b65e0 0x7fab200b8af0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab3c10de10 0x7fab3c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab3c105300 0x7fab3c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 --2- 172.21.15.27:0/2456738363 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fab3c1049b0 0x7fab3c19a490 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 >> 172.21.15.27:0/2456738363 conn(0x7fab3c079720 msgr2=0x7fab3c100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 shutdown_connections 2024-09-21T01:19:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:45.563+0000 7fab43000640 1 -- 172.21.15.27:0/2456738363 wait complete. 2024-09-21T01:19:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:45 smithi027 ceph-mon[30375]: pgmap v318: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:45 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2456738363' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:19:46.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:45 smithi089 ceph-mon[34486]: pgmap v318: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:46.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:45 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2456738363' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:19:46.304 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2024-09-21T01:19:46.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:46.304 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T01:19:46.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi027.stderr:ceph-fuse[76489]: fuse finished with error 0 and tester_r 0 2024-09-21T01:19:46.363 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T01:19:46.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:45 smithi098 ceph-mon[34372]: pgmap v318: 129 pgs: 1 activating, 1 peering, 127 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:46.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:45 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2456738363' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:19:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:47 smithi027 ceph-mon[30375]: pgmap v319: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-21T01:19:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:47 smithi089 ceph-mon[34486]: pgmap v319: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-21T01:19:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:47 smithi098 ceph-mon[34372]: pgmap v319: 129 pgs: 1 activating, 1 peering, 127 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-21T01:19:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:49 smithi027 ceph-mon[30375]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:49 smithi089 ceph-mon[34486]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:49 smithi098 ceph-mon[34372]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:51 smithi027 ceph-mon[30375]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 322 B/s wr, 0 op/s 2024-09-21T01:19:52.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:51 smithi089 ceph-mon[34486]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 322 B/s wr, 0 op/s 2024-09-21T01:19:52.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:51 smithi098 ceph-mon[34372]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 322 B/s wr, 0 op/s 2024-09-21T01:19:52.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:19:52.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:19:52.465 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T01:19:52.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:19:52.904 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:53.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 -- 172.21.15.27:0/2542911640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 msgr2=0x7f68b4077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 --2- 172.21.15.27:0/2542911640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b4077d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009900 tx=0x7f68a802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 -- 172.21.15.27:0/2542911640 shutdown_connections 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 --2- 172.21.15.27:0/2542911640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4114740 0x7f68b4116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 --2- 172.21.15.27:0/2542911640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4078290 0x7f68b4114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 --2- 172.21.15.27:0/2542911640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b4077d50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 -- 172.21.15.27:0/2542911640 >> 172.21.15.27:0/2542911640 conn(0x7f68b4101810 msgr2=0x7f68b4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 -- 172.21.15.27:0/2542911640 shutdown_connections 2024-09-21T01:19:53.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.656+0000 7f68bbe00640 1 -- 172.21.15.27:0/2542911640 wait complete. 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.657+0000 7f68bbe00640 1 Processor -- start 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 -- start start 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b41a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4078290 0x7f68b41a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4114740 0x7f68b41a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f68b411a120 con 0x7f68b4077940 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4119fa0 con 0x7f68b4114740 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68bbe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68b411a2a0 con 0x7f68b4078290 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b41a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:53.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68b8a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4078290 0x7f68b41a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68b9400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b41a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58540/0 (socket says 172.21.15.27:58540) 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.658+0000 7f68b9400640 1 -- 172.21.15.27:0/3172318330 learned_addr learned my addr 172.21.15.27:0/3172318330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9e00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4114740 0x7f68b41a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9400640 1 -- 172.21.15.27:0/3172318330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4078290 msgr2=0x7f68b41a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9400640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4078290 0x7f68b41a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9400640 1 -- 172.21.15.27:0/3172318330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4114740 msgr2=0x7f68b41a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9400640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4114740 0x7f68b41a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9400640 1 -- 172.21.15.27:0/3172318330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b41a6a40 con 0x7f68b4077940 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9e00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4114740 0x7f68b41a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b8a00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4078290 0x7f68b41a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:53.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.659+0000 7f68b9400640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b41a0ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005ab0 tx=0x7f68a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:53.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.660+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a803f070 con 0x7f68b4077940 2024-09-21T01:19:53.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.660+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a8031b60 con 0x7f68b4077940 2024-09-21T01:19:53.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.660+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b41a6cd0 con 0x7f68b4077940 2024-09-21T01:19:53.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.660+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8035030 con 0x7f68b4077940 2024-09-21T01:19:53.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.660+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41ad490 con 0x7f68b4077940 2024-09-21T01:19:53.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.661+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b4077d50 con 0x7f68b4077940 2024-09-21T01:19:53.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.664+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f68a8042070 con 0x7f68b4077940 2024-09-21T01:19:53.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.665+0000 7f68b2000640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f689c0b6560 0x7f689c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:53.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.665+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f68a8108640 con 0x7f68b4077940 2024-09-21T01:19:53.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.666+0000 7f68b8a00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f689c0b6560 0x7f689c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:53.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.666+0000 7f68b8a00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f689c0b6560 0x7f689c0b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f68b407a250 tx=0x7f68a0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:53.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.667+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68a80cea50 con 0x7f68b4077940 2024-09-21T01:19:53.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.867+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68b410f930 con 0x7f68b4077940 2024-09-21T01:19:53.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.868+0000 7f68b2000640 1 -- 172.21.15.27:0/3172318330 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24907 (secure 0 0 0) 0x7f68a80d7030 con 0x7f68b4077940 2024-09-21T01:19:53.870 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:53.870 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:41.886071+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:19:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f689c0b6560 msgr2=0x7f689c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f689c0b6560 0x7f689c0b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f68b407a250 tx=0x7f68a0005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 msgr2=0x7f68b41a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b41a0ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68a8005ab0 tx=0x7f68a8003c10 comp rx=0 tx=0).stop 2024-09-21T01:19:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 shutdown_connections 2024-09-21T01:19:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f689c0b6560 0x7f689c0b8a70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f68b4114740 0x7f68b41a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68b4078290 0x7f68b41a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 --2- 172.21.15.27:0/3172318330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68b4077940 0x7f68b41a0ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 >> 172.21.15.27:0/3172318330 conn(0x7f68b4101810 msgr2=0x7f68b41167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:53.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 shutdown_connections 2024-09-21T01:19:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:53.871+0000 7f68bbe00640 1 -- 172.21.15.27:0/3172318330 wait complete. 2024-09-21T01:19:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:53 smithi027 ceph-mon[30375]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-09-21T01:19:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:53 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3172318330' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:53 smithi089 ceph-mon[34486]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-09-21T01:19:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:53 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3172318330' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:53 smithi098 ceph-mon[34372]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-09-21T01:19:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:53 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3172318330' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:54.648 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:19:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 -- 172.21.15.27:0/2241258566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb4076100 msgr2=0x7fcbb410f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 --2- 172.21.15.27:0/2241258566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb4076100 0x7fcbb410f9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009900 tx=0x7fcb9c02a410 comp rx=0 tx=0).stop 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 -- 172.21.15.27:0/2241258566 shutdown_connections 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 --2- 172.21.15.27:0/2241258566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb410ff10 0x7fcbb4112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 --2- 172.21.15.27:0/2241258566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb4076100 0x7fcbb410f9d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 --2- 172.21.15.27:0/2241258566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 0x7fcbb4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.731+0000 7fcbb9600640 1 -- 172.21.15.27:0/2241258566 >> 172.21.15.27:0/2241258566 conn(0x7fcbb40ff440 msgr2=0x7fcbb4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.732+0000 7fcbb9600640 1 -- 172.21.15.27:0/2241258566 shutdown_connections 2024-09-21T01:19:55.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.732+0000 7fcbb9600640 1 -- 172.21.15.27:0/2241258566 wait complete. 2024-09-21T01:19:55.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 Processor -- start 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 -- start start 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 0x7fcbb419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 0x7fcbb419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb410ff10 0x7fcbb41a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4115ac0 con 0x7fcbb410ff10 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4115940 con 0x7fcbb40757b0 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.733+0000 7fcbb9600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb4115c40 con 0x7fcbb4076100 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbabe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 0x7fcbb419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbabe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 0x7fcbb419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45172/0 (socket says 172.21.15.27:45172) 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbabe00640 1 -- 172.21.15.27:0/754145181 learned_addr learned my addr 172.21.15.27:0/754145181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:55.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb3e00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 0x7fcbb419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:55.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb8c00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb410ff10 0x7fcbb41a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:55.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb8c00640 1 -- 172.21.15.27:0/754145181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 msgr2=0x7fcbb419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb8c00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 0x7fcbb419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb8c00640 1 -- 172.21.15.27:0/754145181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 msgr2=0x7fcbb419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb8c00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 0x7fcbb419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb8c00640 1 -- 172.21.15.27:0/754145181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb41a2480 con 0x7fcbb410ff10 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbabe00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 0x7fcbb419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.734+0000 7fcbb3e00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 0x7fcbb419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.735+0000 7fcbb8c00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb410ff10 0x7fcbb41a1d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009fd0 tx=0x7fcbac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.735+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00ce20 con 0x7fcbb410ff10 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.735+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac005020 con 0x7fcbb410ff10 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.735+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb41a2770 con 0x7fcbb410ff10 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.735+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010690 con 0x7fcbb410ff10 2024-09-21T01:19:55.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.735+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb41a8ed0 con 0x7fcbb410ff10 2024-09-21T01:19:55.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.736+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb4075bc0 con 0x7fcbb410ff10 2024-09-21T01:19:55.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.739+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fcbac0054d0 con 0x7fcbb410ff10 2024-09-21T01:19:55.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.741+0000 7fcbb1600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcb980b6560 0x7fcb980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:55.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.741+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fcbac0e7e70 con 0x7fcbb410ff10 2024-09-21T01:19:55.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.741+0000 7fcbb3e00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcb980b6560 0x7fcb980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:55.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.742+0000 7fcbb3e00640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcb980b6560 0x7fcb980b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009fc0 tx=0x7fcba4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:55.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.742+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcbac0ae2e0 con 0x7fcbb410ff10 2024-09-21T01:19:55.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.943+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbb41a9170 con 0x7fcbb410ff10 2024-09-21T01:19:55.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.945+0000 7fcbb1600640 1 -- 172.21.15.27:0/754145181 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24907 (secure 0 0 0) 0x7fcbac0b26c0 con 0x7fcbb410ff10 2024-09-21T01:19:55.946 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:19:55.946 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:41.886071+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:19:55.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcb980b6560 msgr2=0x7fcb980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:55.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcb980b6560 0x7fcb980b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009fc0 tx=0x7fcba4005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb410ff10 msgr2=0x7fcbb41a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb410ff10 0x7fcbb41a1d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009fd0 tx=0x7fcbac00ef10 comp rx=0 tx=0).stop 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 shutdown_connections 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fcb980b6560 0x7fcb980b8a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb410ff10 0x7fcbb41a1d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbb4076100 0x7fcbb419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 --2- 172.21.15.27:0/754145181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcbb40757b0 0x7fcbb419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 >> 172.21.15.27:0/754145181 conn(0x7fcbb40ff440 msgr2=0x7fcbb4109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 shutdown_connections 2024-09-21T01:19:55.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:55.948+0000 7fcbb9600640 1 -- 172.21.15.27:0/754145181 wait complete. 2024-09-21T01:19:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:55 smithi027 ceph-mon[30375]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:55 smithi089 ceph-mon[34486]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:55 smithi098 ceph-mon[34372]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:56.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T01:19:57.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:56 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/754145181' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:57.236 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:19:57.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:56 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/754145181' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:57.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:56 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/754145181' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:19:57.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 -- 172.21.15.27:0/2771670437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 msgr2=0x7f1940103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:57.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 --2- 172.21.15.27:0/2771670437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f1940103f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1930009900 tx=0x7f193002a400 comp rx=0 tx=0).stop 2024-09-21T01:19:57.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 -- 172.21.15.27:0/2771670437 shutdown_connections 2024-09-21T01:19:57.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 --2- 172.21.15.27:0/2771670437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1940109e10 0x7f1940110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 --2- 172.21.15.27:0/2771670437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19401094c0 0x7f19401098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 --2- 172.21.15.27:0/2771670437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f1940103f90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 -- 172.21.15.27:0/2771670437 >> 172.21.15.27:0/2771670437 conn(0x7f19400ff5d0 msgr2=0x7f1940101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 -- 172.21.15.27:0/2771670437 shutdown_connections 2024-09-21T01:19:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.861+0000 7f1947a00640 1 -- 172.21.15.27:0/2771670437 wait complete. 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.862+0000 7f1947a00640 1 Processor -- start 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.862+0000 7f1947a00640 1 -- start start 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1947a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f194019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1947a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19401094c0 0x7f194019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1947a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1940109e10 0x7f194019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1947a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1940113ad0 con 0x7f1940109e10 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1947a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1940113950 con 0x7f1940103b80 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1947a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1940113c50 con 0x7f19401094c0 2024-09-21T01:19:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1945000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f194019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1945000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f194019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51076/0 (socket says 172.21.15.27:51076) 2024-09-21T01:19:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1945000640 1 -- 172.21.15.27:0/3222448859 learned_addr learned my addr 172.21.15.27:0/3222448859 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:19:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1945a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1940109e10 0x7f194019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.863+0000 7f1937e00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19401094c0 0x7f194019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f1945000640 1 -- 172.21.15.27:0/3222448859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19401094c0 msgr2=0x7f194019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f1945000640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19401094c0 0x7f194019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f1945000640 1 -- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1940109e10 msgr2=0x7f194019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f1945000640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1940109e10 0x7f194019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f1945000640 1 -- 172.21.15.27:0/3222448859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19401a03f0 con 0x7f1940103b80 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f1945000640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f194019a480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1930005940 tx=0x7f19300040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.864+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193003f030 con 0x7f1940103b80 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.865+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19300044c0 con 0x7f1940103b80 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.865+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930035070 con 0x7f1940103b80 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.865+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401a0680 con 0x7f1940103b80 2024-09-21T01:19:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.865+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401a6e40 con 0x7f1940103b80 2024-09-21T01:19:57.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.866+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1940104e00 con 0x7f1940103b80 2024-09-21T01:19:57.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.868+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f1930042030 con 0x7f1940103b80 2024-09-21T01:19:57.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.870+0000 7f193e000640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f19240b65e0 0x7f19240b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:19:57.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.870+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f1930107d00 con 0x7f1940103b80 2024-09-21T01:19:57.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.870+0000 7f1937e00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f19240b65e0 0x7f19240b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:19:57.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.871+0000 7f1937e00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f19240b65e0 0x7f19240b8af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1940109030 tx=0x7f1928005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:19:57.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:57.872+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19300ce110 con 0x7f1940103b80 2024-09-21T01:19:58.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.114+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f194010bbe0 con 0x7f1940103b80 2024-09-21T01:19:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:57 smithi089 ceph-mon[34486]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:57 smithi098 ceph-mon[34372]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:57 smithi027 ceph-mon[30375]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-21T01:19:58.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.946+0000 7f193e000640 1 -- 172.21.15.27:0/3222448859 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f19300d24f0 con 0x7f1940103b80 2024-09-21T01:19:58.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f19240b65e0 msgr2=0x7f19240b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f19240b65e0 0x7f19240b8af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1940109030 tx=0x7f1928005f90 comp rx=0 tx=0).stop 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 msgr2=0x7f194019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f194019a480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1930005940 tx=0x7f19300040c0 comp rx=0 tx=0).stop 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 shutdown_connections 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f19240b65e0 0x7f19240b8af0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1940109e10 0x7f194019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19401094c0 0x7f194019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 --2- 172.21.15.27:0/3222448859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1940103b80 0x7f194019a480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 >> 172.21.15.27:0/3222448859 conn(0x7f19400ff5d0 msgr2=0x7f19401019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 shutdown_connections 2024-09-21T01:19:58.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:19:58.950+0000 7f1947a00640 1 -- 172.21.15.27:0/3222448859 wait complete. 2024-09-21T01:19:59.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:58 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3222448859' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:19:59.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:58 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:19:59.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:58 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3222448859' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:19:59.322 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:58 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:19:59.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:58 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3222448859' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:19:59.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:58 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T01:19:59.742 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-21T01:19:59.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T01:20:00.128 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:00.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:59 smithi027 ceph-mon[30375]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T01:20:00.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:59 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T01:20:00.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:19:59 smithi027 ceph-mon[30375]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:20:00.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:59 smithi089 ceph-mon[34486]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T01:20:00.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:59 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T01:20:00.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:19:59 smithi089 ceph-mon[34486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:20:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:59 smithi098 ceph-mon[34372]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T01:20:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:59 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T01:20:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:19:59 smithi098 ceph-mon[34372]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 -- 172.21.15.27:0/162033033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf34105320 msgr2=0x7fbf3410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/162033033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf34105320 0x7fbf3410d870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009900 tx=0x7fbf2002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 -- 172.21.15.27:0/162033033 shutdown_connections 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/162033033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf3410de30 0x7fbf34110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/162033033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf34105320 0x7fbf3410d870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/162033033 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf341049d0 0x7fbf34104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 -- 172.21.15.27:0/162033033 >> 172.21.15.27:0/162033033 conn(0x7fbf34079740 msgr2=0x7fbf34079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 -- 172.21.15.27:0/162033033 shutdown_connections 2024-09-21T01:20:00.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.764+0000 7fbf3aa00640 1 -- 172.21.15.27:0/162033033 wait complete. 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.765+0000 7fbf3aa00640 1 Processor -- start 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.765+0000 7fbf3aa00640 1 -- start start 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf341049d0 0x7fbf34102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 0x7fbf34102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf3410de30 0x7fbf34103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf3aa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113a90 con 0x7fbf341049d0 2024-09-21T01:20:00.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf3aa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113910 con 0x7fbf34105320 2024-09-21T01:20:00.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf3aa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34113c10 con 0x7fbf3410de30 2024-09-21T01:20:00.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf341049d0 0x7fbf34102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 0x7fbf34102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf341049d0 0x7fbf34102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44140/0 (socket says 172.21.15.27:44140) 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 0x7fbf34102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:51090/0 (socket says 172.21.15.27:51090) 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33e00640 1 -- 172.21.15.27:0/3377071626 learned_addr learned my addr 172.21.15.27:0/3377071626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf38a00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf3410de30 0x7fbf34103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33400640 1 -- 172.21.15.27:0/3377071626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf3410de30 msgr2=0x7fbf34103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.766+0000 7fbf33400640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf3410de30 0x7fbf34103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf33400640 1 -- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf341049d0 msgr2=0x7fbf34102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf33400640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf341049d0 0x7fbf34102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf33400640 1 -- 172.21.15.27:0/3377071626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf341a8780 con 0x7fbf34105320 2024-09-21T01:20:00.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf33400640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 0x7fbf34102c50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf2002a910 tx=0x7fbf20031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2003f070 con 0x7fbf34105320 2024-09-21T01:20:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf341a8a10 con 0x7fbf34105320 2024-09-21T01:20:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf20004490 con 0x7fbf34105320 2024-09-21T01:20:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.767+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20035070 con 0x7fbf34105320 2024-09-21T01:20:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.768+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf341a8ef0 con 0x7fbf34105320 2024-09-21T01:20:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.769+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3406c240 con 0x7fbf34105320 2024-09-21T01:20:00.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.772+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7fbf2003b460 con 0x7fbf34105320 2024-09-21T01:20:00.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.774+0000 7fbf30c00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbf180b6560 0x7fbf180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:00.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.774+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fbf20107ed0 con 0x7fbf34105320 2024-09-21T01:20:00.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.774+0000 7fbf33e00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbf180b6560 0x7fbf180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:00.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.775+0000 7fbf33e00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbf180b6560 0x7fbf180b8a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf34104160 tx=0x7fbf24005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:00.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:00.776+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf200d26c0 con 0x7fbf34105320 2024-09-21T01:20:01.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.023+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fbf34103a10 con 0x7fbf34105320 2024-09-21T01:20:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:00 smithi089 ceph-mon[34486]: overall HEALTH_OK 2024-09-21T01:20:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:00 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:20:01.346 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:00 smithi098 ceph-mon[34372]: overall HEALTH_OK 2024-09-21T01:20:01.347 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:00 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:20:01.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.350+0000 7fbf30c00640 1 -- 172.21.15.27:0/3377071626 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fbf34103a10 con 0x7fbf34105320 2024-09-21T01:20:01.351 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:00 smithi027 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T01:20:01.351 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:00 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:20:01.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbf180b6560 msgr2=0x7fbf180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:01.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbf180b6560 0x7fbf180b8a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf34104160 tx=0x7fbf24005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:01.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 msgr2=0x7fbf34102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:01.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 0x7fbf34102c50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf2002a910 tx=0x7fbf20031a20 comp rx=0 tx=0).stop 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 shutdown_connections 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbf180b6560 0x7fbf180b8a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf3410de30 0x7fbf34103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbf34105320 0x7fbf34102c50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 --2- 172.21.15.27:0/3377071626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf341049d0 0x7fbf34102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.353+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 >> 172.21.15.27:0/3377071626 conn(0x7fbf34079740 msgr2=0x7fbf34106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.354+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 shutdown_connections 2024-09-21T01:20:01.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:01.354+0000 7fbf3aa00640 1 -- 172.21.15.27:0/3377071626 wait complete. 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3377071626' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T01:20:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-21T01:20:02.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:01 smithi027 ceph-mon[30375]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3377071626' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-21T01:20:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:01 smithi089 ceph-mon[34486]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3377071626' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-21T01:20:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:01 smithi098 ceph-mon[34372]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-21T01:20:02.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T01:20:03.319 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:03.467 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] up:active 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T01:20:03.468 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:03 smithi027 ceph-mon[30375]: pgmap v327: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] up:active 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T01:20:03.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:03 smithi089 ceph-mon[34486]: pgmap v327: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] up:active 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-21T01:20:03.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-09-21T01:20:03.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-21T01:20:03.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:03 smithi098 ceph-mon[34372]: pgmap v327: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 1 op/s 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.098+0000 7f92cd200640 1 -- 172.21.15.27:0/2329288940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c8105d90 msgr2=0x7f92c81061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.098+0000 7f92cd200640 1 --2- 172.21.15.27:0/2329288940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c8105d90 0x7f92c81061a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009900 tx=0x7f92b802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 -- 172.21.15.27:0/2329288940 shutdown_connections 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 --2- 172.21.15.27:0/2329288940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c810c020 0x7f92c8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 --2- 172.21.15.27:0/2329288940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c810b6d0 0x7f92c810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 --2- 172.21.15.27:0/2329288940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c8105d90 0x7f92c81061a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 -- 172.21.15.27:0/2329288940 >> 172.21.15.27:0/2329288940 conn(0x7f92c8101860 msgr2=0x7f92c8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 -- 172.21.15.27:0/2329288940 shutdown_connections 2024-09-21T01:20:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.099+0000 7f92cd200640 1 -- 172.21.15.27:0/2329288940 wait complete. 2024-09-21T01:20:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 Processor -- start 2024-09-21T01:20:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 -- start start 2024-09-21T01:20:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 0x7f92c819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c810b6d0 0x7f92c819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c810c020 0x7f92c81a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8115ce0 con 0x7f92c810b6d0 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8115b60 con 0x7f92c810c020 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.100+0000 7f92cd200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8115e60 con 0x7f92c8105d90 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 0x7f92c819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 0x7f92c819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:42876/0 (socket says 172.21.15.27:42876) 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 -- 172.21.15.27:0/892393082 learned_addr learned my addr 172.21.15.27:0/892393082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c6800640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c810c020 0x7f92c81a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5400640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c810b6d0 0x7f92c819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 -- 172.21.15.27:0/892393082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c810c020 msgr2=0x7f92c81a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c810c020 0x7f92c81a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 -- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c810b6d0 msgr2=0x7f92c819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c810b6d0 0x7f92c819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.101+0000 7f92c5e00640 1 -- 172.21.15.27:0/892393082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c81a2660 con 0x7f92c8105d90 2024-09-21T01:20:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.102+0000 7f92c5e00640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 0x7f92c819c6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92b8005940 tx=0x7f92b80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.102+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b803f030 con 0x7f92c8105d90 2024-09-21T01:20:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.102+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c81a28f0 con 0x7f92c8105d90 2024-09-21T01:20:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.102+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b80044c0 con 0x7f92c8105d90 2024-09-21T01:20:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.102+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8035070 con 0x7f92c8105d90 2024-09-21T01:20:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.102+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81a90b0 con 0x7f92c8105d90 2024-09-21T01:20:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.103+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c81061a0 con 0x7f92c8105d90 2024-09-21T01:20:04.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.108+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f92b8042030 con 0x7f92c8105d90 2024-09-21T01:20:04.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.110+0000 7f92bea00640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f92ac0b6560 0x7f92ac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:04.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.110+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f92b8107d60 con 0x7f92c8105d90 2024-09-21T01:20:04.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.110+0000 7f92c5400640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f92ac0b6560 0x7f92ac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:04.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.110+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92b80c9d40 con 0x7f92c8105d90 2024-09-21T01:20:04.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.111+0000 7f92c5400640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f92ac0b6560 0x7f92ac0b8a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104f60 tx=0x7f92b4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:04.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:04.371+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f92c802e640 con 0x7f92c8105d90 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] up:active 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:04.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:04 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:04.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] up:active 2024-09-21T01:20:04.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T01:20:04.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T01:20:04.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-09-21T01:20:04.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-09-21T01:20:04.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:04.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:04 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] up:active 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:creating} 8 up:standby 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:04 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:05.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.386+0000 7f92bea00640 1 -- 172.21.15.27:0/892393082 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f92b80ce1d0 con 0x7f92c8105d90 2024-09-21T01:20:05.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f92ac0b6560 msgr2=0x7f92ac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f92ac0b6560 0x7f92ac0b8a70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f92c8104f60 tx=0x7f92b4005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 msgr2=0x7f92c819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 0x7f92c819c6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92b8005940 tx=0x7f92b80040c0 comp rx=0 tx=0).stop 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 shutdown_connections 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f92ac0b6560 0x7f92ac0b8a70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f92c810c020 0x7f92c81a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c810b6d0 0x7f92c819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 --2- 172.21.15.27:0/892393082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c8105d90 0x7f92c819c6f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.389+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 >> 172.21.15.27:0/892393082 conn(0x7f92c8101860 msgr2=0x7f92c810ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.390+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 shutdown_connections 2024-09-21T01:20:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:05.390+0000 7f92cd200640 1 -- 172.21.15.27:0/892393082 wait complete. 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] up:active 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: Cluster is now healthy 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/892393082' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:20:05.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T01:20:05.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:20:05.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-21T01:20:05.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: pgmap v328: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2024-09-21T01:20:05.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:20:05.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:20:05.649 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] up:active 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: Cluster is now healthy 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/892393082' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:20:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-21T01:20:05.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: pgmap v328: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2024-09-21T01:20:05.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:20:05.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:20:05.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] up:active 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: Cluster is now healthy 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=h=up:active,3=i=up:active} 8 up:standby 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/892393082' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:20:05.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T01:20:05.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T01:20:05.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-21T01:20:05.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: pgmap v328: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2024-09-21T01:20:05.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:20:05.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:20:05.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:20:06.207 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T01:20:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:06 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T01:20:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:06 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] up:active 2024-09-21T01:20:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:06 smithi027 ceph-mon[30375]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T01:20:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:06 smithi027 ceph-mon[30375]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:20:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:06.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:06 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T01:20:06.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:06 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] up:active 2024-09-21T01:20:06.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:06 smithi098 ceph-mon[34372]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T01:20:06.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:06 smithi098 ceph-mon[34372]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:20:06.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:06 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T01:20:06.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:06 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] up:active 2024-09-21T01:20:06.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:06 smithi089 ceph-mon[34486]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T01:20:06.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:06 smithi089 ceph-mon[34486]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:20:07.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 -- 172.21.15.27:0/2913205651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810b6d0 msgr2=0x7f016810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:07.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 --2- 172.21.15.27:0/2913205651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810b6d0 0x7f016810bae0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0150009860 tx=0x7f015002a340 comp rx=0 tx=0).stop 2024-09-21T01:20:07.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 -- 172.21.15.27:0/2913205651 shutdown_connections 2024-09-21T01:20:07.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 --2- 172.21.15.27:0/2913205651 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f016810c020 0x7f0168112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:07.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 --2- 172.21.15.27:0/2913205651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810b6d0 0x7f016810bae0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:07.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 --2- 172.21.15.27:0/2913205651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0168105d90 0x7f01681061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:07.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 -- 172.21.15.27:0/2913205651 >> 172.21.15.27:0/2913205651 conn(0x7f0168101860 msgr2=0x7f0168103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:07.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 -- 172.21.15.27:0/2913205651 shutdown_connections 2024-09-21T01:20:07.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.315+0000 7f016dc00640 1 -- 172.21.15.27:0/2913205651 wait complete. 2024-09-21T01:20:07.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.316+0000 7f016dc00640 1 Processor -- start 2024-09-21T01:20:07.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.316+0000 7f016dc00640 1 -- start start 2024-09-21T01:20:07.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f016dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 0x7f016819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:07.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f016dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f016810b6d0 0x7f016819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:07.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f016dc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810c020 0x7f01681a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:07.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f016dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0168115ce0 con 0x7f0168105d90 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f016dc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0168115b60 con 0x7f016810b6d0 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f016dc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0168115e60 con 0x7f016810c020 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0166a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 0x7f016819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0167400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810c020 0x7f01681a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0166a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 0x7f016819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45042/0 (socket says 172.21.15.27:45042) 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0166a00640 1 -- 172.21.15.27:0/2743261230 learned_addr learned my addr 172.21.15.27:0/2743261230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f015fe00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f016810b6d0 0x7f016819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0167400640 1 -- 172.21.15.27:0/2743261230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f016810b6d0 msgr2=0x7f016819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0167400640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f016810b6d0 0x7f016819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0167400640 1 -- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 msgr2=0x7f016819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.317+0000 7f0167400640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 0x7f016819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f0167400640 1 -- 172.21.15.27:0/2743261230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01681a2700 con 0x7f016810c020 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f0166a00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 0x7f016819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:07.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f0167400640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810c020 0x7f01681a1fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f016000efc0 tx=0x7f016000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:07.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160003970 con 0x7f016810c020 2024-09-21T01:20:07.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0160004020 con 0x7f016810c020 2024-09-21T01:20:07.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01681a29f0 con 0x7f016810c020 2024-09-21T01:20:07.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016000b940 con 0x7f016810c020 2024-09-21T01:20:07.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.318+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a9150 con 0x7f016810c020 2024-09-21T01:20:07.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.319+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01681061a0 con 0x7f016810c020 2024-09-21T01:20:07.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.322+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f016001d020 con 0x7f016810c020 2024-09-21T01:20:07.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.323+0000 7f015f400640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f014c0b6560 0x7f014c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:07.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.324+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f01600e7c30 con 0x7f016810c020 2024-09-21T01:20:07.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.324+0000 7f0166a00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f014c0b6560 0x7f014c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:07.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.324+0000 7f0166a00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f014c0b6560 0x7f014c0b8a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0158011500 tx=0x7f0158005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:07.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.325+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01600ae040 con 0x7f016810c020 2024-09-21T01:20:07.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:07.566+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f016810ddf0 con 0x7f016810c020 2024-09-21T01:20:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:07 smithi098 ceph-mon[34372]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 8 op/s 2024-09-21T01:20:07.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:07 smithi027 ceph-mon[30375]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 8 op/s 2024-09-21T01:20:07.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:07 smithi089 ceph-mon[34486]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 8 op/s 2024-09-21T01:20:08.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.410+0000 7f015f400640 1 -- 172.21.15.27:0/2743261230 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f01600b2420 con 0x7f016810c020 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f014c0b6560 msgr2=0x7f014c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f014c0b6560 0x7f014c0b8a70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0158011500 tx=0x7f0158005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810c020 msgr2=0x7f01681a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810c020 0x7f01681a1fd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f016000efc0 tx=0x7f016000c800 comp rx=0 tx=0).stop 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 shutdown_connections 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f014c0b6560 0x7f014c0b8a70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f016810c020 0x7f01681a1fd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f016810b6d0 0x7f016819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 --2- 172.21.15.27:0/2743261230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0168105d90 0x7f016819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 >> 172.21.15.27:0/2743261230 conn(0x7f0168101860 msgr2=0x7f0168103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 shutdown_connections 2024-09-21T01:20:08.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:08.413+0000 7f016dc00640 1 -- 172.21.15.27:0/2743261230 wait complete. 2024-09-21T01:20:08.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:08 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2743261230' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:20:08.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:08 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:20:08.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:08 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2743261230' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:20:08.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:08 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:20:08.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:08 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2743261230' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:20:08.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:08 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T01:20:09.230 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-21T01:20:09.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T01:20:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:09.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:09 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T01:20:09.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:09 smithi027 ceph-mon[30375]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:20:09.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:09 smithi027 ceph-mon[30375]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-09-21T01:20:09.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:09 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T01:20:09.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:09 smithi089 ceph-mon[34486]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:20:09.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:09 smithi089 ceph-mon[34486]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-09-21T01:20:09.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:09 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T01:20:09.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:09 smithi098 ceph-mon[34372]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:20:09.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:09 smithi098 ceph-mon[34372]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.3 KiB/s wr, 14 op/s 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1477433564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc1092d0 msgr2=0x7f0ecc1096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1477433564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc1096e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009900 tx=0x7f0eb802a410 comp rx=0 tx=0).stop 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1477433564 shutdown_connections 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1477433564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ecc109c20 0x7f0ecc110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1477433564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc1096e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1477433564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc103990 0x7f0ecc103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1477433564 >> 172.21.15.27:0/1477433564 conn(0x7f0ecc0ff3e0 msgr2=0x7f0ecc101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:10.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1477433564 shutdown_connections 2024-09-21T01:20:10.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.219+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1477433564 wait complete. 2024-09-21T01:20:10.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.220+0000 7f0ed0a00640 1 Processor -- start 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.220+0000 7f0ed0a00640 1 -- start start 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ed0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ecc103990 0x7f0ecc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ed0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ed0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc109c20 0x7f0ecc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ed0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc1138e0 con 0x7f0ecc109c20 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ed0a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc113760 con 0x7f0ecc103990 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ed0a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc113a60 con 0x7f0ecc1092d0 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60310/0 (socket says 172.21.15.27:60310) 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 -- 172.21.15.27:0/1271145013 learned_addr learned my addr 172.21.15.27:0/1271145013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ecb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc109c20 0x7f0ecc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0eca800640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ecc103990 0x7f0ecc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 -- 172.21.15.27:0/1271145013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ecc103990 msgr2=0x7f0ecc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ecc103990 0x7f0ecc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 -- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc109c20 msgr2=0x7f0ecc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc109c20 0x7f0ecc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.221+0000 7f0ec9e00640 1 -- 172.21.15.27:0/1271145013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc1a85b0 con 0x7f0ecc1092d0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.222+0000 7f0ec9e00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc075c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0eb802a930 tx=0x7f0eb8031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.222+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb803f030 con 0x7f0ecc1092d0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.222+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc1a8840 con 0x7f0ecc1092d0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.222+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc1a8d70 con 0x7f0ecc1092d0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.222+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb8004490 con 0x7f0ecc1092d0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.222+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8049060 con 0x7f0ecc1092d0 2024-09-21T01:20:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.224+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc103da0 con 0x7f0ecc1092d0 2024-09-21T01:20:10.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.226+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 154529+0+0 (secure 0 0 0) 0x7f0eb8042030 con 0x7f0ecc1092d0 2024-09-21T01:20:10.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.228+0000 7f0ec3400640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0eb00b6560 0x7f0eb00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:10.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.228+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f0eb8107750 con 0x7f0ecc1092d0 2024-09-21T01:20:10.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.228+0000 7f0eca800640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0eb00b6560 0x7f0eb00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:10.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.228+0000 7f0eca800640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0eb00b6560 0x7f0eb00b8a70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc077160 tx=0x7f0ebc005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:10.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.229+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0eb80d1f40 con 0x7f0ecc1092d0 2024-09-21T01:20:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.459+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0ecc063800 con 0x7f0eb00b6560 2024-09-21T01:20:10.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.494+0000 7f0ec3400640 1 -- 172.21.15.27:0/1271145013 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ecc063800 con 0x7f0eb00b6560 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0eb00b6560 msgr2=0x7f0eb00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0eb00b6560 0x7f0eb00b8a70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc077160 tx=0x7f0ebc005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 msgr2=0x7f0ecc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc075c50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0eb802a930 tx=0x7f0eb8031a20 comp rx=0 tx=0).stop 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 shutdown_connections 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0eb00b6560 0x7f0eb00b8a70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ecc109c20 0x7f0ecc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ecc1092d0 0x7f0ecc075c50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 --2- 172.21.15.27:0/1271145013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ecc103990 0x7f0ecc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 >> 172.21.15.27:0/1271145013 conn(0x7f0ecc0ff3e0 msgr2=0x7f0ecc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 shutdown_connections 2024-09-21T01:20:10.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:10.497+0000 7f0ed0a00640 1 -- 172.21.15.27:0/1271145013 wait complete. 2024-09-21T01:20:10.853 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:20:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:20:10.467+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:20:10.853 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:20:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:20:10.467+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:20:10.853 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:20:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:20:10.467+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:20:10.853 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:20:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:20:10.467+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:20:10.853 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:20:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:20:10.467+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:20:11.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T01:20:11.658 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:11.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:11 smithi089 ceph-mon[34486]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-09-21T01:20:11.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:11 smithi089 ceph-mon[34486]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:11.798 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:11 smithi027 ceph-mon[30375]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-09-21T01:20:11.798 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:11 smithi027 ceph-mon[30375]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:11.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:11 smithi098 ceph-mon[34372]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-09-21T01:20:11.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:11 smithi098 ceph-mon[34372]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:12.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 -- 172.21.15.27:0/2611438796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010b6c0 msgr2=0x7f87f010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:12.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 --2- 172.21.15.27:0/2611438796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010b6c0 0x7f87f010bad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009900 tx=0x7f87d802a410 comp rx=0 tx=0).stop 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 -- 172.21.15.27:0/2611438796 shutdown_connections 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 --2- 172.21.15.27:0/2611438796 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f010c010 0x7f87f0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 --2- 172.21.15.27:0/2611438796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010b6c0 0x7f87f010bad0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 --2- 172.21.15.27:0/2611438796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f0105d80 0x7f87f0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.321+0000 7f87f7a00640 1 -- 172.21.15.27:0/2611438796 >> 172.21.15.27:0/2611438796 conn(0x7f87f01017d0 msgr2=0x7f87f0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.322+0000 7f87f7a00640 1 -- 172.21.15.27:0/2611438796 shutdown_connections 2024-09-21T01:20:12.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.322+0000 7f87f7a00640 1 -- 172.21.15.27:0/2611438796 wait complete. 2024-09-21T01:20:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 Processor -- start 2024-09-21T01:20:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 -- start start 2024-09-21T01:20:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f0105d80 0x7f87f019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f010b6c0 0x7f87f019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010c010 0x7f87f01a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0115cd0 con 0x7f87f010c010 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0115b50 con 0x7f87f0105d80 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.323+0000 7f87f7a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0115e50 con 0x7f87f010b6c0 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87e7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f010b6c0 0x7f87f019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f0105d80 0x7f87f019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010c010 0x7f87f01a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87e7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f010b6c0 0x7f87f019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60344/0 (socket says 172.21.15.27:60344) 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f0105d80 0x7f87f019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52252/0 (socket says 172.21.15.27:52252) 2024-09-21T01:20:12.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87e7e00640 1 -- 172.21.15.27:0/618768276 learned_addr learned my addr 172.21.15.27:0/618768276 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 -- 172.21.15.27:0/618768276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f010b6c0 msgr2=0x7f87f019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:12.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f010b6c0 0x7f87f019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 -- 172.21.15.27:0/618768276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f0105d80 msgr2=0x7f87f019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f0105d80 0x7f87f019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 -- 172.21.15.27:0/618768276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f01a26b0 con 0x7f87f010c010 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.324+0000 7f87f5a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010c010 0x7f87f01a1f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87e800da10 tx=0x7f87e800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.325+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800bd70 con 0x7f87f010c010 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.325+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e8003e20 con 0x7f87f010c010 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.325+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f01a29a0 con 0x7f87f010c010 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.325+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80106c0 con 0x7f87f010c010 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.325+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f01a9100 con 0x7f87f010c010 2024-09-21T01:20:12.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.326+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f0106190 con 0x7f87f010c010 2024-09-21T01:20:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.329+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f87e8004270 con 0x7f87f010c010 2024-09-21T01:20:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.331+0000 7f87ee000640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87d40b65f0 0x7f87d40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.332+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f87e80e80a0 con 0x7f87f010c010 2024-09-21T01:20:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.332+0000 7f87f5000640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87d40b65f0 0x7f87d40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.332+0000 7f87f5000640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87d40b65f0 0x7f87d40b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009bd0 tx=0x7f87e0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.334+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f87e80ae350 con 0x7f87f010c010 2024-09-21T01:20:12.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.561+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f87f0063800 con 0x7f87d40b65f0 2024-09-21T01:20:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.592+0000 7f87ee000640 1 -- 172.21.15.27:0/618768276 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f87f0063800 con 0x7f87d40b65f0 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87d40b65f0 msgr2=0x7f87d40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87d40b65f0 0x7f87d40b8b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009bd0 tx=0x7f87e0005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010c010 msgr2=0x7f87f01a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010c010 0x7f87f01a1f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87e800da10 tx=0x7f87e800ded0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 shutdown_connections 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f87d40b65f0 0x7f87d40b8b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87f010c010 0x7f87f01a1f80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87f010b6c0 0x7f87f019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 --2- 172.21.15.27:0/618768276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f87f0105d80 0x7f87f019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 >> 172.21.15.27:0/618768276 conn(0x7f87f01017d0 msgr2=0x7f87f0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 shutdown_connections 2024-09-21T01:20:12.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:12.594+0000 7f87f7a00640 1 -- 172.21.15.27:0/618768276 wait complete. 2024-09-21T01:20:12.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:12 smithi098 ceph-mon[34372]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T01:20:12.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:12 smithi027 ceph-mon[30375]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T01:20:13.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:12 smithi089 ceph-mon[34486]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T01:20:13.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T01:20:13.799 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:13 smithi098 ceph-mon[34372]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.1 KiB/s wr, 20 op/s 2024-09-21T01:20:13.799 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:13 smithi098 ceph-mon[34372]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:13.859 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:13.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:13 smithi027 ceph-mon[30375]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.1 KiB/s wr, 20 op/s 2024-09-21T01:20:13.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:13 smithi027 ceph-mon[30375]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:14.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:13 smithi089 ceph-mon[34486]: pgmap v332: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.1 KiB/s wr, 20 op/s 2024-09-21T01:20:14.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:13 smithi089 ceph-mon[34486]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:14.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.533+0000 7f36a0a00640 1 -- 172.21.15.27:0/1717986802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c105bb0 msgr2=0x7f369c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.533+0000 7f36a0a00640 1 --2- 172.21.15.27:0/1717986802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c105bb0 0x7f369c105fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f368c009900 tx=0x7f368c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 -- 172.21.15.27:0/1717986802 shutdown_connections 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 --2- 172.21.15.27:0/1717986802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c10be40 0x7f369c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 --2- 172.21.15.27:0/1717986802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10b4f0 0x7f369c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 --2- 172.21.15.27:0/1717986802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c105bb0 0x7f369c105fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 -- 172.21.15.27:0/1717986802 >> 172.21.15.27:0/1717986802 conn(0x7f369c101680 msgr2=0x7f369c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 -- 172.21.15.27:0/1717986802 shutdown_connections 2024-09-21T01:20:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.534+0000 7f36a0a00640 1 -- 172.21.15.27:0/1717986802 wait complete. 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.535+0000 7f36a0a00640 1 Processor -- start 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.535+0000 7f36a0a00640 1 -- start start 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.535+0000 7f36a0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 0x7f369c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.535+0000 7f36a0a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 0x7f369c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f36a0a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10be40 0x7f369c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f36a0a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f369c115b50 con 0x7f369c10b4f0 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f36a0a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1159d0 con 0x7f369c10be40 2024-09-21T01:20:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f36a0a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f369c115cd0 con 0x7f369c105bb0 2024-09-21T01:20:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f3699e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 0x7f369c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 0x7f369c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f3699e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 0x7f369c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45124/0 (socket says 172.21.15.27:45124) 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f3699e00640 1 -- 172.21.15.27:0/3050712690 learned_addr learned my addr 172.21.15.27:0/3050712690 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 0x7f369c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60362/0 (socket says 172.21.15.27:60362) 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369b200640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10be40 0x7f369c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 -- 172.21.15.27:0/3050712690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10be40 msgr2=0x7f369c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10be40 0x7f369c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 -- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 msgr2=0x7f369c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 0x7f369c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369a800640 1 -- 172.21.15.27:0/3050712690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1a2490 con 0x7f369c105bb0 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f3699e00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 0x7f369c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.536+0000 7f369b200640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10be40 0x7f369c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:20:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.537+0000 7f369a800640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 0x7f369c19c520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f368c02a9e0 tx=0x7f368c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.537+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c03f070 con 0x7f369c105bb0 2024-09-21T01:20:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.537+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1a2720 con 0x7f369c105bb0 2024-09-21T01:20:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.537+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368c031b50 con 0x7f369c105bb0 2024-09-21T01:20:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.537+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c035030 con 0x7f369c105bb0 2024-09-21T01:20:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.537+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1a8ee0 con 0x7f369c105bb0 2024-09-21T01:20:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.538+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c106e30 con 0x7f369c105bb0 2024-09-21T01:20:14.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.541+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f368c042070 con 0x7f369c105bb0 2024-09-21T01:20:14.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.542+0000 7f3693400640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f36800b6670 0x7f36800b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:14.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.542+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f368c108100 con 0x7f369c105bb0 2024-09-21T01:20:14.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.542+0000 7f3699e00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f36800b6670 0x7f36800b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:14.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.543+0000 7f3699e00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f36800b6670 0x7f36800b8b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f369c10aed0 tx=0x7f3688005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:14.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.544+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f368c0ce3b0 con 0x7f369c105bb0 2024-09-21T01:20:14.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.773+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f369c063800 con 0x7f36800b6670 2024-09-21T01:20:14.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.780+0000 7f3693400640 1 -- 172.21.15.27:0/3050712690 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f369c063800 con 0x7f36800b6670 2024-09-21T01:20:14.781 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_0/ca040ac3-ab58-440b-896d-f85d8a01bff7 2024-09-21T01:20:14.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.782+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f36800b6670 msgr2=0x7f36800b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:14.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.782+0000 7f36a0a00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f36800b6670 0x7f36800b8b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f369c10aed0 tx=0x7f3688005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:14.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.782+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 msgr2=0x7f369c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:14.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.782+0000 7f36a0a00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 0x7f369c19c520 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f368c02a9e0 tx=0x7f368c003b90 comp rx=0 tx=0).stop 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 shutdown_connections 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f36800b6670 0x7f36800b8b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f369c10be40 0x7f369c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f369c10b4f0 0x7f369c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 --2- 172.21.15.27:0/3050712690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c105bb0 0x7f369c19c520 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 >> 172.21.15.27:0/3050712690 conn(0x7f369c101680 msgr2=0x7f369c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 shutdown_connections 2024-09-21T01:20:14.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:14.783+0000 7f36a0a00640 1 -- 172.21.15.27:0/3050712690 wait complete. 2024-09-21T01:20:15.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T01:20:15.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:15 smithi027 ceph-mon[30375]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-09-21T01:20:15.777 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:15 smithi027 ceph-mon[30375]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:15 smithi098 ceph-mon[34372]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-09-21T01:20:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:15 smithi098 ceph-mon[34372]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:15 smithi089 ceph-mon[34486]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-09-21T01:20:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:15 smithi089 ceph-mon[34486]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:16.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 -- 172.21.15.27:0/1084057361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 msgr2=0x7f443c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:16.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 --2- 172.21.15.27:0/1084057361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c103f70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f442c009900 tx=0x7f442c02a410 comp rx=0 tx=0).stop 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 -- 172.21.15.27:0/1084057361 shutdown_connections 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 --2- 172.21.15.27:0/1084057361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c109df0 0x7f443c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 --2- 172.21.15.27:0/1084057361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c1094a0 0x7f443c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 --2- 172.21.15.27:0/1084057361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c103f70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.786+0000 7f4442200640 1 -- 172.21.15.27:0/1084057361 >> 172.21.15.27:0/1084057361 conn(0x7f443c07b850 msgr2=0x7f443c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.787+0000 7f4442200640 1 -- 172.21.15.27:0/1084057361 shutdown_connections 2024-09-21T01:20:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.787+0000 7f4442200640 1 -- 172.21.15.27:0/1084057361 wait complete. 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 Processor -- start 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 -- start start 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c1094a0 0x7f443c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 0x7f443c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f443c113b00 con 0x7f443c103b60 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f443c113980 con 0x7f443c1094a0 2024-09-21T01:20:16.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.788+0000 7f4442200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f443c113c80 con 0x7f443c109df0 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 0x7f443c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c1094a0 0x7f443c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 0x7f443c1a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60460/0 (socket says 172.21.15.27:60460) 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 -- 172.21.15.27:0/2056919842 learned_addr learned my addr 172.21.15.27:0/2056919842 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35994/0 (socket says 172.21.15.27:35994) 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 -- 172.21.15.27:0/2056919842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c1094a0 msgr2=0x7f443c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c1094a0 0x7f443c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 -- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 msgr2=0x7f443c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:16.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443be00640 1 -- 172.21.15.27:0/2056919842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c1a2640 con 0x7f443c109df0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443aa00640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c1094a0 0x7f443c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.789+0000 7f443b400640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.790+0000 7f443be00640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 0x7f443c1a1f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f443400eb10 tx=0x7f443400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.790+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434004b20 con 0x7f443c109df0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.790+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1a2930 con 0x7f443c109df0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.790+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4434004cc0 con 0x7f443c109df0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.790+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434010670 con 0x7f443c109df0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.790+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c1a9090 con 0x7f443c109df0 2024-09-21T01:20:16.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.791+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c103f70 con 0x7f443c109df0 2024-09-21T01:20:16.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.794+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f4434005020 con 0x7f443c109df0 2024-09-21T01:20:16.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.795+0000 7f4433e00640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44200b65f0 0x7f44200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.795+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f44340e81d0 con 0x7f443c109df0 2024-09-21T01:20:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.795+0000 7f443b400640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44200b65f0 0x7f44200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:16.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.796+0000 7f443b400640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44200b65f0 0x7f44200b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f442c005ab0 tx=0x7f442c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:16.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:16.797+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44340ae480 con 0x7f443c109df0 2024-09-21T01:20:17.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.022+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f443c1079c0 con 0x7f44200b65f0 2024-09-21T01:20:17.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.057+0000 7f4433e00640 1 -- 172.21.15.27:0/2056919842 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f443c1079c0 con 0x7f44200b65f0 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44200b65f0 msgr2=0x7f44200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44200b65f0 0x7f44200b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f442c005ab0 tx=0x7f442c005a20 comp rx=0 tx=0).stop 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 msgr2=0x7f443c1a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 0x7f443c1a1f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f443400eb10 tx=0x7f443400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 shutdown_connections 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f44200b65f0 0x7f44200b8b00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:17.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f443c109df0 0x7f443c1a1f10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f443c1094a0 0x7f443c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 --2- 172.21.15.27:0/2056919842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f443c103b60 0x7f443c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.059+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 >> 172.21.15.27:0/2056919842 conn(0x7f443c07b850 msgr2=0x7f443c1072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.060+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 shutdown_connections 2024-09-21T01:20:17.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:17.060+0000 7f4442200640 1 -- 172.21.15.27:0/2056919842 wait complete. 2024-09-21T01:20:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:17 smithi098 ceph-mon[34372]: pgmap v334: 129 pgs: 129 active+clean; 614 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-21T01:20:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:17 smithi098 ceph-mon[34372]: from='client.24955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:17 smithi027 ceph-mon[30375]: pgmap v334: 129 pgs: 129 active+clean; 614 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-21T01:20:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:17 smithi027 ceph-mon[30375]: from='client.24955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:17.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T01:20:17.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:17 smithi089 ceph-mon[34486]: pgmap v334: 129 pgs: 129 active+clean; 614 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-21T01:20:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:17 smithi089 ceph-mon[34486]: from='client.24955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 -- 172.21.15.27:0/870771865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04103b60 msgr2=0x7f0e04103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 --2- 172.21.15.27:0/870771865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04103b60 0x7f0e04103f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009900 tx=0x7f0df402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 -- 172.21.15.27:0/870771865 shutdown_connections 2024-09-21T01:20:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 --2- 172.21.15.27:0/870771865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04109df0 0x7f0e04110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 --2- 172.21.15.27:0/870771865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 0x7f0e041098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 --2- 172.21.15.27:0/870771865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04103b60 0x7f0e04103f70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 -- 172.21.15.27:0/870771865 >> 172.21.15.27:0/870771865 conn(0x7f0e0407b850 msgr2=0x7f0e0407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 -- 172.21.15.27:0/870771865 shutdown_connections 2024-09-21T01:20:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.015+0000 7f0e09c00640 1 -- 172.21.15.27:0/870771865 wait complete. 2024-09-21T01:20:19.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.016+0000 7f0e09c00640 1 Processor -- start 2024-09-21T01:20:19.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.016+0000 7f0e09c00640 1 -- start start 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.016+0000 7f0e09c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 0x7f0e0419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e09c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 0x7f0e0419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e09c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04109df0 0x7f0e041a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e09c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e04113ab0 con 0x7f0e041094a0 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e09c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e04113930 con 0x7f0e04103b60 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e09c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0e04113c30 con 0x7f0e04109df0 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 0x7f0e0419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 0x7f0e0419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 0x7f0e0419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52982/0 (socket says 172.21.15.27:52982) 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e03400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04109df0 0x7f0e041a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 -- 172.21.15.27:0/215788464 learned_addr learned my addr 172.21.15.27:0/215788464 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 0x7f0e0419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36012/0 (socket says 172.21.15.27:36012) 2024-09-21T01:20:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 -- 172.21.15.27:0/215788464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04109df0 msgr2=0x7f0e041a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04109df0 0x7f0e041a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 -- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 msgr2=0x7f0e0419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 0x7f0e0419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.017+0000 7f0e02a00640 1 -- 172.21.15.27:0/215788464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e041a2730 con 0x7f0e04103b60 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.018+0000 7f0e02a00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 0x7f0e0419c7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005940 tx=0x7f0df40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.018+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df403f030 con 0x7f0e04103b60 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.018+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e041a29c0 con 0x7f0e04103b60 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.018+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0df40044c0 con 0x7f0e04103b60 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.018+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4035070 con 0x7f0e04103b60 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.018+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e041a9180 con 0x7f0e04103b60 2024-09-21T01:20:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.019+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e04103f70 con 0x7f0e04103b60 2024-09-21T01:20:19.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.022+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0df4042030 con 0x7f0e04103b60 2024-09-21T01:20:19.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.023+0000 7f0dfb400640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0de80b65f0 0x7f0de80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:19.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.024+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f0df4107f40 con 0x7f0e04103b60 2024-09-21T01:20:19.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.024+0000 7f0e02000640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0de80b65f0 0x7f0de80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:19.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.025+0000 7f0e02000640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0de80b65f0 0x7f0de80b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e04108fe0 tx=0x7f0df0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:19.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.026+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0df40d25d0 con 0x7f0e04103b60 2024-09-21T01:20:19.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.256+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0e04107b00 con 0x7f0de80b65f0 2024-09-21T01:20:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.261+0000 7f0dfb400640 1 -- 172.21.15.27:0/215788464 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0e04107b00 con 0x7f0de80b65f0 2024-09-21T01:20:19.261 INFO:teuthology.orchestra.run.smithi027.stdout:/volumes/qa/sv_1/467e0d3e-aecf-42d9-88a8-be95e03ed056 2024-09-21T01:20:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.263+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0de80b65f0 msgr2=0x7f0de80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.263+0000 7f0e09c00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0de80b65f0 0x7f0de80b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e04108fe0 tx=0x7f0df0005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.263+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 msgr2=0x7f0e0419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.263+0000 7f0e09c00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 0x7f0e0419c7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0df4005940 tx=0x7f0df40040c0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 shutdown_connections 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0de80b65f0 0x7f0de80b8b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e04109df0 0x7f0e041a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e041094a0 0x7f0e0419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 --2- 172.21.15.27:0/215788464 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e04103b60 0x7f0e0419c7c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 >> 172.21.15.27:0/215788464 conn(0x7f0e0407b850 msgr2=0x7f0e041073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 shutdown_connections 2024-09-21T01:20:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:19.264+0000 7f0e09c00640 1 -- 172.21.15.27:0/215788464 wait complete. 2024-09-21T01:20:19.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:19 smithi098 ceph-mon[34372]: pgmap v335: 129 pgs: 129 active+clean; 614 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-09-21T01:20:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:19 smithi027 ceph-mon[30375]: pgmap v335: 129 pgs: 129 active+clean; 614 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-09-21T01:20:19.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:19 smithi089 ceph-mon[34486]: pgmap v335: 129 pgs: 129 active+clean; 614 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-09-21T01:20:20.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:20:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:20.782 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:20 smithi027 ceph-mon[30375]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:20.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:20 smithi098 ceph-mon[34372]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:20.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:20 smithi089 ceph-mon[34486]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:20:21.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 -- 172.21.15.27:0/1375799238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010b6c0 msgr2=0x7f1c0010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 --2- 172.21.15.27:0/1375799238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010b6c0 0x7f1c0010bad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009900 tx=0x7f1bec02a410 comp rx=0 tx=0).stop 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 -- 172.21.15.27:0/1375799238 shutdown_connections 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 --2- 172.21.15.27:0/1375799238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c0010c010 0x7f1c00112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 --2- 172.21.15.27:0/1375799238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010b6c0 0x7f1c0010bad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 --2- 172.21.15.27:0/1375799238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c00106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 -- 172.21.15.27:0/1375799238 >> 172.21.15.27:0/1375799238 conn(0x7f1c00101810 msgr2=0x7f1c00103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 -- 172.21.15.27:0/1375799238 shutdown_connections 2024-09-21T01:20:21.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.074+0000 7f1c07c00640 1 -- 172.21.15.27:0/1375799238 wait complete. 2024-09-21T01:20:21.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.075+0000 7f1c07c00640 1 Processor -- start 2024-09-21T01:20:21.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 -- start start 2024-09-21T01:20:21.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c0019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:21.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c0010b6c0 0x7f1c0019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:21.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010c010 0x7f1c001a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00115cd0 con 0x7f1c0010c010 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00115b50 con 0x7f1c00105d80 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c07c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00115e50 con 0x7f1c0010b6c0 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c05200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c0019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c05200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c0019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53012/0 (socket says 172.21.15.27:53012) 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c05c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010c010 0x7f1c001a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.076+0000 7f1c05200640 1 -- 172.21.15.27:0/4184625261 learned_addr learned my addr 172.21.15.27:0/4184625261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:21.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1bffe00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c0010b6c0 0x7f1c0019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1c05200640 1 -- 172.21.15.27:0/4184625261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c0010b6c0 msgr2=0x7f1c0019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1c05200640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c0010b6c0 0x7f1c0019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1c05200640 1 -- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010c010 msgr2=0x7f1c001a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1c05200640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010c010 0x7f1c001a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1c05200640 1 -- 172.21.15.27:0/4184625261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c001a2630 con 0x7f1c00105d80 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.077+0000 7f1c05200640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c0019c6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400ea60 tx=0x7f1bf400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.078+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400cde0 con 0x7f1c00105d80 2024-09-21T01:20:21.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.078+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c001a2920 con 0x7f1c00105d80 2024-09-21T01:20:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.078+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf40035f0 con 0x7f1c00105d80 2024-09-21T01:20:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.078+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4010690 con 0x7f1c00105d80 2024-09-21T01:20:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.078+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001a9080 con 0x7f1c00105d80 2024-09-21T01:20:21.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.079+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c00106190 con 0x7f1c00105d80 2024-09-21T01:20:21.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.081+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1bf40037a0 con 0x7f1c00105d80 2024-09-21T01:20:21.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.083+0000 7f1bfd600640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1be80b6670 0x7f1be80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.083+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f1bf40e7dc0 con 0x7f1c00105d80 2024-09-21T01:20:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.083+0000 7f1bffe00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1be80b6670 0x7f1be80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:21.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.084+0000 7f1bffe00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1be80b6670 0x7f1be80b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009230 tx=0x7f1bec031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:21.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.085+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1bf4056710 con 0x7f1c00105d80 2024-09-21T01:20:21.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.325+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c0019d7f0 con 0x7f1c00105d80 2024-09-21T01:20:21.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.326+0000 7f1bfd600640 1 -- 172.21.15.27:0/4184625261 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f1bf40ae070 con 0x7f1c00105d80 2024-09-21T01:20:21.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:21.327 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T01:20:08:397428+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:20:07.570104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24718,24730,24734,24752,24776]},"id":1}]} 2024-09-21T01:20:21.327 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:20:21.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1be80b6670 msgr2=0x7f1be80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:21.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1be80b6670 0x7f1be80b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1bec009230 tx=0x7f1bec031400 comp rx=0 tx=0).stop 2024-09-21T01:20:21.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 msgr2=0x7f1c0019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:21.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c0019c6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400ea60 tx=0x7f1bf400ef20 comp rx=0 tx=0).stop 2024-09-21T01:20:21.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 shutdown_connections 2024-09-21T01:20:21.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1be80b6670 0x7f1be80b8b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c0010c010 0x7f1c001a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c0010b6c0 0x7f1c0019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 --2- 172.21.15.27:0/4184625261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c00105d80 0x7f1c0019c6c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:21.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 >> 172.21.15.27:0/4184625261 conn(0x7f1c00101810 msgr2=0x7f1c00103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:21.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 shutdown_connections 2024-09-21T01:20:21.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:21.328+0000 7f1c07c00640 1 -- 172.21.15.27:0/4184625261 wait complete. 2024-09-21T01:20:21.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:21 smithi027 ceph-mon[30375]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-09-21T01:20:21.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4184625261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:20:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:21 smithi098 ceph-mon[34372]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-09-21T01:20:21.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4184625261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:20:22.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:21 smithi089 ceph-mon[34486]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-09-21T01:20:22.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4184625261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:20:22.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:20:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 -- 172.21.15.27:0/2741251701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc105d80 msgr2=0x7f9fdc106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2741251701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc105d80 0x7f9fdc106190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009900 tx=0x7f9fd002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 -- 172.21.15.27:0/2741251701 shutdown_connections 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2741251701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fdc10c010 0x7f9fdc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2741251701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2741251701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc105d80 0x7f9fdc106190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 -- 172.21.15.27:0/2741251701 >> 172.21.15.27:0/2741251701 conn(0x7f9fdc101810 msgr2=0x7f9fdc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 -- 172.21.15.27:0/2741251701 shutdown_connections 2024-09-21T01:20:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.177+0000 7f9fe4000640 1 -- 172.21.15.27:0/2741251701 wait complete. 2024-09-21T01:20:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.178+0000 7f9fe4000640 1 Processor -- start 2024-09-21T01:20:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.178+0000 7f9fe4000640 1 -- start start 2024-09-21T01:20:23.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe4000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fdc105d80 0x7f9fdc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe4000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe4000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc10c010 0x7f9fdc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe4000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc115cd0 con 0x7f9fdc10c010 2024-09-21T01:20:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe4000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc115b50 con 0x7f9fdc105d80 2024-09-21T01:20:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe4000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc115e50 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc10c010 0x7f9fdc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe0c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60524/0 (socket says 172.21.15.27:60524) 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe0c00640 1 -- 172.21.15.27:0/2865684627 learned_addr learned my addr 172.21.15.27:0/2865684627 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe2000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc10c010 0x7f9fdc1a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36038/0 (socket says 172.21.15.27:36038) 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.179+0000 7f9fe1600640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fdc105d80 0x7f9fdc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe0c00640 1 -- 172.21.15.27:0/2865684627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fdc105d80 msgr2=0x7f9fdc19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:23.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe0c00640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fdc105d80 0x7f9fdc19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe0c00640 1 -- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc10c010 msgr2=0x7f9fdc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe0c00640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc10c010 0x7f9fdc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe0c00640 1 -- 172.21.15.27:0/2865684627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc1a2650 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe0c00640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc19cc80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800e9c0 tx=0x7f9fc800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800ccc0 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc80054d0 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc1a2940 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8010430 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.180+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc1a90a0 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.181+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc106190 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.184+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9fc8005000 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.186+0000 7f9fda000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9fc40b6670 0x7f9fc40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:23.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.186+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f9fc80e7e30 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.186+0000 7f9fe1600640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9fc40b6670 0x7f9fc40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:23.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.187+0000 7f9fe1600640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9fc40b6670 0x7f9fc40b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0031440 tx=0x7f9fd0009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:23.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.187+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9fc8056710 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.385+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fdc19d810 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.387+0000 7f9fda000640 1 -- 172.21.15.27:0/2865684627 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24907 (secure 0 0 0) 0x7f9fdc19d810 con 0x7f9fdc10b6c0 2024-09-21T01:20:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:23.388 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:41.886071+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:20:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9fc40b6670 msgr2=0x7f9fc40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9fc40b6670 0x7f9fc40b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0031440 tx=0x7f9fd0009560 comp rx=0 tx=0).stop 2024-09-21T01:20:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 msgr2=0x7f9fdc19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc19cc80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800e9c0 tx=0x7f9fc800ee80 comp rx=0 tx=0).stop 2024-09-21T01:20:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 shutdown_connections 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9fc40b6670 0x7f9fc40b8b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc10c010 0x7f9fdc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9fdc10b6c0 0x7f9fdc19cc80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.389+0000 7f9fe4000640 1 --2- 172.21.15.27:0/2865684627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9fdc105d80 0x7f9fdc19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.390+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 >> 172.21.15.27:0/2865684627 conn(0x7f9fdc101810 msgr2=0x7f9fdc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.390+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 shutdown_connections 2024-09-21T01:20:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:23.390+0000 7f9fe4000640 1 -- 172.21.15.27:0/2865684627 wait complete. 2024-09-21T01:20:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:23 smithi027 ceph-mon[30375]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-21T01:20:23.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:23 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2865684627' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:23.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:23 smithi098 ceph-mon[34372]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-21T01:20:23.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:23 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2865684627' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:24.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:23 smithi089 ceph-mon[34486]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-21T01:20:24.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:23 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2865684627' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:24.307 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T01:20:24.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:20:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 -- 172.21.15.27:0/1654704237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824105320 msgr2=0x7fa82410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 --2- 172.21.15.27:0/1654704237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824105320 0x7fa82410d870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009900 tx=0x7fa80c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 -- 172.21.15.27:0/1654704237 shutdown_connections 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 --2- 172.21.15.27:0/1654704237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa82410de30 0x7fa824110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 --2- 172.21.15.27:0/1654704237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824105320 0x7fa82410d870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 --2- 172.21.15.27:0/1654704237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa8241049d0 0x7fa824104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 -- 172.21.15.27:0/1654704237 >> 172.21.15.27:0/1654704237 conn(0x7fa824079740 msgr2=0x7fa824079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 -- 172.21.15.27:0/1654704237 shutdown_connections 2024-09-21T01:20:25.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.344+0000 7fa828e00640 1 -- 172.21.15.27:0/1654704237 wait complete. 2024-09-21T01:20:25.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.345+0000 7fa828e00640 1 Processor -- start 2024-09-21T01:20:25.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 -- start start 2024-09-21T01:20:25.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8241049d0 0x7fa82419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:25.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa824105320 0x7fa82419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:25.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 0x7fa8241a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:25.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa824113a90 con 0x7fa8241049d0 2024-09-21T01:20:25.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa824113910 con 0x7fa82410de30 2024-09-21T01:20:25.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa828e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa824113c10 con 0x7fa824105320 2024-09-21T01:20:25.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa822800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 0x7fa8241a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:25.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa81be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa824105320 0x7fa82419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:25.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.346+0000 7fa822800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 0x7fa8241a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:53058/0 (socket says 172.21.15.27:53058) 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 -- 172.21.15.27:0/1866282821 learned_addr learned my addr 172.21.15.27:0/1866282821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa821e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8241049d0 0x7fa82419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 -- 172.21.15.27:0/1866282821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa824105320 msgr2=0x7fa82419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa824105320 0x7fa82419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 -- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8241049d0 msgr2=0x7fa82419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8241049d0 0x7fa82419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 -- 172.21.15.27:0/1866282821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8241a2650 con 0x7fa82410de30 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.347+0000 7fa822800640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 0x7fa8241a1f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00eb10 tx=0x7fa81c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.348+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004b20 con 0x7fa82410de30 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.348+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8241a2940 con 0x7fa82410de30 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.348+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa81c004cc0 con 0x7fa82410de30 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.348+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c010690 con 0x7fa82410de30 2024-09-21T01:20:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.348+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a90a0 con 0x7fa82410de30 2024-09-21T01:20:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.349+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82406c240 con 0x7fa82410de30 2024-09-21T01:20:25.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.352+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa81c010830 con 0x7fa82410de30 2024-09-21T01:20:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.353+0000 7fa81aa00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa8080b6670 0x7fa8080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.353+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fa81c0e7fe0 con 0x7fa82410de30 2024-09-21T01:20:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.353+0000 7fa821e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa8080b6670 0x7fa8080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.354+0000 7fa821e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa8080b6670 0x7fa8080b8b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa82406c940 tx=0x7fa814005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.355+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa81c056710 con 0x7fa82410de30 2024-09-21T01:20:25.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.552+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa824100d60 con 0x7fa8080b6670 2024-09-21T01:20:25.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.560+0000 7fa81aa00640 1 -- 172.21.15.27:0/1866282821 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340389 (secure 0 0 0) 0x7fa824100d60 con 0x7fa8080b6670 2024-09-21T01:20:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:25.566 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:20:25.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa8080b6670 msgr2=0x7fa8080b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:25.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa8080b6670 0x7fa8080b8b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa82406c940 tx=0x7fa814005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 msgr2=0x7fa8241a1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 0x7fa8241a1f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00eb10 tx=0x7fa81c00c2d0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 shutdown_connections 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa8080b6670 0x7fa8080b8b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa82410de30 0x7fa8241a1f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa824105320 0x7fa82419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 --2- 172.21.15.27:0/1866282821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8241049d0 0x7fa82419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 >> 172.21.15.27:0/1866282821 conn(0x7fa824079740 msgr2=0x7fa824100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 shutdown_connections 2024-09-21T01:20:25.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:25.565+0000 7fa828e00640 1 -- 172.21.15.27:0/1866282821 wait complete. 2024-09-21T01:20:25.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:25 smithi098 ceph-mon[34372]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-21T01:20:25.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:25 smithi027 ceph-mon[30375]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-21T01:20:26.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:25 smithi089 ceph-mon[34486]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-09-21T01:20:26.507 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-21T01:20:24.437801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654350,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":282,"num_read_kb":242,"num_write":369,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371592,"kb_used_data":14024,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570232,"statfs":{"total":1151940427776,"available":1151559917568,"internally_reserved":0,"allocated":14360576,"data_stored":8124032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53327,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":114,"num_write":36,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003892"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:22.426474+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:20:22.426474+0000","last_peered":"2024-09-21T01:20:22.426474+0000","last_clean":"2024-09-21T01:20:22.426474+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:20:22.426474+0000","last_undegraded":"2024-09-21T01:20:22.426474+0000","last_fullsized":"2024-09-21T01:20:22.426474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.902018+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:19:41.902018+0000","last_peered":"2024-09-21T01:19:41.902018+0000","last_clean":"2024-09-21T01:19:41.902018+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:19:41.902018+0000","last_undegraded":"2024-09-21T01:19:41.902018+0000","last_fullsized":"2024-09-21T01:19:41.902018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896721+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:19:41.896721+0000","last_peered":"2024-09-21T01:19:41.896721+0000","last_clean":"2024-09-21T01:19:41.896721+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:19:41.896721+0000","last_undegraded":"2024-09-21T01:19:41.896721+0000","last_fullsized":"2024-09-21T01:19:41.896721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.104024+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:19:42.104024+0000","last_peered":"2024-09-21T01:19:42.104024+0000","last_clean":"2024-09-21T01:19:42.104024+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:19:42.104024+0000","last_undegraded":"2024-09-21T01:19:42.104024+0000","last_fullsized":"2024-09-21T01:19:42.104024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893193+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:19:41.893193+0000","last_peered":"2024-09-21T01:19:41.893193+0000","last_clean":"2024-09-21T01:19:41.893193+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:19:41.893193+0000","last_undegraded":"2024-09-21T01:19:41.893193+0000","last_fullsized":"2024-09-21T01:19:41.893193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892871+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:19:41.892871+0000","last_peered":"2024-09-21T01:19:41.892871+0000","last_clean":"2024-09-21T01:19:41.892871+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:19:41.892871+0000","last_undegraded":"2024-09-21T01:19:41.892871+0000","last_fullsized":"2024-09-21T01:19:41.892871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:05.397635+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:20:05.397635+0000","last_peered":"2024-09-21T01:20:05.397635+0000","last_clean":"2024-09-21T01:20:05.397635+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:20:05.397635+0000","last_undegraded":"2024-09-21T01:20:05.397635+0000","last_fullsized":"2024-09-21T01:20:05.397635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968924+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:19:41.968924+0000","last_peered":"2024-09-21T01:19:41.968924+0000","last_clean":"2024-09-21T01:19:41.968924+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:19:41.968924+0000","last_undegraded":"2024-09-21T01:19:41.968924+0000","last_fullsized":"2024-09-21T01:19:41.968924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893328+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:19:41.893328+0000","last_peered":"2024-09-21T01:19:41.893328+0000","last_clean":"2024-09-21T01:19:41.893328+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:19:41.893328+0000","last_undegraded":"2024-09-21T01:19:41.893328+0000","last_fullsized":"2024-09-21T01:19:41.893328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386940+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:04.386940+0000","last_peered":"2024-09-21T01:20:04.386940+0000","last_clean":"2024-09-21T01:20:04.386940+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:04.386940+0000","last_undegraded":"2024-09-21T01:20:04.386940+0000","last_fullsized":"2024-09-21T01:20:04.386940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.179169+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:19:42.179169+0000","last_peered":"2024-09-21T01:19:42.179169+0000","last_clean":"2024-09-21T01:19:42.179169+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:19:42.179169+0000","last_undegraded":"2024-09-21T01:19:42.179169+0000","last_fullsized":"2024-09-21T01:19:42.179169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892994+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:19:41.892994+0000","last_peered":"2024-09-21T01:19:41.892994+0000","last_clean":"2024-09-21T01:19:41.892994+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:19:41.892994+0000","last_undegraded":"2024-09-21T01:19:41.892994+0000","last_fullsized":"2024-09-21T01:19:41.892994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890856+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:19:41.890856+0000","last_peered":"2024-09-21T01:19:41.890856+0000","last_clean":"2024-09-21T01:19:41.890856+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:19:41.890856+0000","last_undegraded":"2024-09-21T01:19:41.890856+0000","last_fullsized":"2024-09-21T01:19:41.890856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:01.359747+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:01.359747+0000","last_peered":"2024-09-21T01:20:01.359747+0000","last_clean":"2024-09-21T01:20:01.359747+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:01.359747+0000","last_undegraded":"2024-09-21T01:20:01.359747+0000","last_fullsized":"2024-09-21T01:20:01.359747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893205+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:19:41.893205+0000","last_peered":"2024-09-21T01:19:41.893205+0000","last_clean":"2024-09-21T01:19:41.893205+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:19:41.893205+0000","last_undegraded":"2024-09-21T01:19:41.893205+0000","last_fullsized":"2024-09-21T01:19:41.893205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103086+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:19:42.103086+0000","last_peered":"2024-09-21T01:19:42.103086+0000","last_clean":"2024-09-21T01:19:42.103086+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:19:42.103086+0000","last_undegraded":"2024-09-21T01:19:42.103086+0000","last_fullsized":"2024-09-21T01:19:42.103086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.379872+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:03.379872+0000","last_peered":"2024-09-21T01:20:03.379872+0000","last_clean":"2024-09-21T01:20:03.379872+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:03.379872+0000","last_undegraded":"2024-09-21T01:20:03.379872+0000","last_fullsized":"2024-09-21T01:20:03.379872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898484+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:19:41.898484+0000","last_peered":"2024-09-21T01:19:41.898484+0000","last_clean":"2024-09-21T01:19:41.898484+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:19:41.898484+0000","last_undegraded":"2024-09-21T01:19:41.898484+0000","last_fullsized":"2024-09-21T01:19:41.898484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.894872+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:19:41.894872+0000","last_peered":"2024-09-21T01:19:41.894872+0000","last_clean":"2024-09-21T01:19:41.894872+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:19:41.894872+0000","last_undegraded":"2024-09-21T01:19:41.894872+0000","last_fullsized":"2024-09-21T01:19:41.894872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386175+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:04.386175+0000","last_peered":"2024-09-21T01:20:04.386175+0000","last_clean":"2024-09-21T01:20:04.386175+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:04.386175+0000","last_undegraded":"2024-09-21T01:20:04.386175+0000","last_fullsized":"2024-09-21T01:20:04.386175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898591+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:19:41.898591+0000","last_peered":"2024-09-21T01:19:41.898591+0000","last_clean":"2024-09-21T01:19:41.898591+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:19:41.898591+0000","last_undegraded":"2024-09-21T01:19:41.898591+0000","last_fullsized":"2024-09-21T01:19:41.898591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890147+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:19:41.890147+0000","last_peered":"2024-09-21T01:19:41.890147+0000","last_clean":"2024-09-21T01:19:41.890147+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:19:41.890147+0000","last_undegraded":"2024-09-21T01:19:41.890147+0000","last_fullsized":"2024-09-21T01:19:41.890147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:01.360664+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:01.360664+0000","last_peered":"2024-09-21T01:20:01.360664+0000","last_clean":"2024-09-21T01:20:01.360664+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:01.360664+0000","last_undegraded":"2024-09-21T01:20:01.360664+0000","last_fullsized":"2024-09-21T01:20:01.360664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893346+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:19:41.893346+0000","last_peered":"2024-09-21T01:19:41.893346+0000","last_clean":"2024-09-21T01:19:41.893346+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:19:41.893346+0000","last_undegraded":"2024-09-21T01:19:41.893346+0000","last_fullsized":"2024-09-21T01:19:41.893346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893111+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:19:41.893111+0000","last_peered":"2024-09-21T01:19:41.893111+0000","last_clean":"2024-09-21T01:19:41.893111+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:19:41.893111+0000","last_undegraded":"2024-09-21T01:19:41.893111+0000","last_fullsized":"2024-09-21T01:19:41.893111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.387246+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:04.387246+0000","last_peered":"2024-09-21T01:20:04.387246+0000","last_clean":"2024-09-21T01:20:04.387246+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:04.387246+0000","last_undegraded":"2024-09-21T01:20:04.387246+0000","last_fullsized":"2024-09-21T01:20:04.387246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896923+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:19:41.896923+0000","last_peered":"2024-09-21T01:19:41.896923+0000","last_clean":"2024-09-21T01:19:41.896923+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:19:41.896923+0000","last_undegraded":"2024-09-21T01:19:41.896923+0000","last_fullsized":"2024-09-21T01:19:41.896923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103705+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:19:42.103705+0000","last_peered":"2024-09-21T01:19:42.103705+0000","last_clean":"2024-09-21T01:19:42.103705+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:19:42.103705+0000","last_undegraded":"2024-09-21T01:19:42.103705+0000","last_fullsized":"2024-09-21T01:19:42.103705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:20.747971+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:20:20.747971+0000","last_peered":"2024-09-21T01:20:20.747971+0000","last_clean":"2024-09-21T01:20:20.747971+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:20:20.747971+0000","last_undegraded":"2024-09-21T01:20:20.747971+0000","last_fullsized":"2024-09-21T01:20:20.747971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.102599+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:19:42.102599+0000","last_peered":"2024-09-21T01:19:42.102599+0000","last_clean":"2024-09-21T01:19:42.102599+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:19:42.102599+0000","last_undegraded":"2024-09-21T01:19:42.102599+0000","last_fullsized":"2024-09-21T01:19:42.102599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177889+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:19:42.177889+0000","last_peered":"2024-09-21T01:19:42.177889+0000","last_clean":"2024-09-21T01:19:42.177889+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:19:42.177889+0000","last_undegraded":"2024-09-21T01:19:42.177889+0000","last_fullsized":"2024-09-21T01:19:42.177889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.387739+0000","last_change":"2024-09-21T01:19:03.692864+0000","last_active":"2024-09-21T01:20:04.387739+0000","last_peered":"2024-09-21T01:20:04.387739+0000","last_clean":"2024-09-21T01:20:04.387739+0000","last_became_active":"2024-09-21T01:19:03.692585+0000","last_became_peered":"2024-09-21T01:19:03.692585+0000","last_unstale":"2024-09-21T01:20:04.387739+0000","last_undegraded":"2024-09-21T01:20:04.387739+0000","last_fullsized":"2024-09-21T01:20:04.387739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:53.429693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895152+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:19:41.895152+0000","last_peered":"2024-09-21T01:19:41.895152+0000","last_clean":"2024-09-21T01:19:41.895152+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:19:41.895152+0000","last_undegraded":"2024-09-21T01:19:41.895152+0000","last_fullsized":"2024-09-21T01:19:41.895152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896304+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:19:41.896304+0000","last_peered":"2024-09-21T01:19:41.896304+0000","last_clean":"2024-09-21T01:19:41.896304+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:19:41.896304+0000","last_undegraded":"2024-09-21T01:19:41.896304+0000","last_fullsized":"2024-09-21T01:19:41.896304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899130+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:19:41.899130+0000","last_peered":"2024-09-21T01:19:41.899130+0000","last_clean":"2024-09-21T01:19:41.899130+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:19:41.899130+0000","last_undegraded":"2024-09-21T01:19:41.899130+0000","last_fullsized":"2024-09-21T01:19:41.899130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890147+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:19:41.890147+0000","last_peered":"2024-09-21T01:19:41.890147+0000","last_clean":"2024-09-21T01:19:41.890147+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:19:41.890147+0000","last_undegraded":"2024-09-21T01:19:41.890147+0000","last_fullsized":"2024-09-21T01:19:41.890147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898803+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:19:41.898803+0000","last_peered":"2024-09-21T01:19:41.898803+0000","last_clean":"2024-09-21T01:19:41.898803+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:19:41.898803+0000","last_undegraded":"2024-09-21T01:19:41.898803+0000","last_fullsized":"2024-09-21T01:19:41.898803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:18.420937+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:20:18.420937+0000","last_peered":"2024-09-21T01:20:18.420937+0000","last_clean":"2024-09-21T01:20:18.420937+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:20:18.420937+0000","last_undegraded":"2024-09-21T01:20:18.420937+0000","last_fullsized":"2024-09-21T01:20:18.420937+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177611+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:19:42.177611+0000","last_peered":"2024-09-21T01:19:42.177611+0000","last_clean":"2024-09-21T01:19:42.177611+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:19:42.177611+0000","last_undegraded":"2024-09-21T01:19:42.177611+0000","last_fullsized":"2024-09-21T01:19:42.177611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.900102+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:19:41.900102+0000","last_peered":"2024-09-21T01:19:41.900102+0000","last_clean":"2024-09-21T01:19:41.900102+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:19:41.900102+0000","last_undegraded":"2024-09-21T01:19:41.900102+0000","last_fullsized":"2024-09-21T01:19:41.900102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:22.426436+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:20:22.426436+0000","last_peered":"2024-09-21T01:20:22.426436+0000","last_clean":"2024-09-21T01:20:22.426436+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:20:22.426436+0000","last_undegraded":"2024-09-21T01:20:22.426436+0000","last_fullsized":"2024-09-21T01:20:22.426436+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.178078+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:19:42.178078+0000","last_peered":"2024-09-21T01:19:42.178078+0000","last_clean":"2024-09-21T01:19:42.178078+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:19:42.178078+0000","last_undegraded":"2024-09-21T01:19:42.178078+0000","last_fullsized":"2024-09-21T01:19:42.178078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177246+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:19:42.177246+0000","last_peered":"2024-09-21T01:19:42.177246+0000","last_clean":"2024-09-21T01:19:42.177246+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:19:42.177246+0000","last_undegraded":"2024-09-21T01:19:42.177246+0000","last_fullsized":"2024-09-21T01:19:42.177246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:02.368655+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:02.368655+0000","last_peered":"2024-09-21T01:20:02.368655+0000","last_clean":"2024-09-21T01:20:02.368655+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:02.368655+0000","last_undegraded":"2024-09-21T01:20:02.368655+0000","last_fullsized":"2024-09-21T01:20:02.368655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.180220+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:19:42.180220+0000","last_peered":"2024-09-21T01:19:42.180220+0000","last_clean":"2024-09-21T01:19:42.180220+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:19:42.180220+0000","last_undegraded":"2024-09-21T01:19:42.180220+0000","last_fullsized":"2024-09-21T01:19:42.180220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.971047+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:19:41.971047+0000","last_peered":"2024-09-21T01:19:41.971047+0000","last_clean":"2024-09-21T01:19:41.971047+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:19:41.971047+0000","last_undegraded":"2024-09-21T01:19:41.971047+0000","last_fullsized":"2024-09-21T01:19:41.971047+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:05.396996+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:05.396996+0000","last_peered":"2024-09-21T01:20:05.396996+0000","last_clean":"2024-09-21T01:20:05.396996+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:05.396996+0000","last_undegraded":"2024-09-21T01:20:05.396996+0000","last_fullsized":"2024-09-21T01:20:05.396996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890786+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:19:41.890786+0000","last_peered":"2024-09-21T01:19:41.890786+0000","last_clean":"2024-09-21T01:19:41.890786+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:19:41.890786+0000","last_undegraded":"2024-09-21T01:19:41.890786+0000","last_fullsized":"2024-09-21T01:19:41.890786+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891338+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:19:41.891338+0000","last_peered":"2024-09-21T01:19:41.891338+0000","last_clean":"2024-09-21T01:19:41.891338+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:19:41.891338+0000","last_undegraded":"2024-09-21T01:19:41.891338+0000","last_fullsized":"2024-09-21T01:19:41.891338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891420+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:19:41.891420+0000","last_peered":"2024-09-21T01:19:41.891420+0000","last_clean":"2024-09-21T01:19:41.891420+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:19:41.891420+0000","last_undegraded":"2024-09-21T01:19:41.891420+0000","last_fullsized":"2024-09-21T01:19:41.891420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:20.424866+0000","last_change":"2024-09-21T01:19:04.020705+0000","last_active":"2024-09-21T01:20:20.424866+0000","last_peered":"2024-09-21T01:20:20.424866+0000","last_clean":"2024-09-21T01:20:20.424866+0000","last_became_active":"2024-09-21T01:19:04.020234+0000","last_became_peered":"2024-09-21T01:19:04.020234+0000","last_unstale":"2024-09-21T01:20:20.424866+0000","last_undegraded":"2024-09-21T01:20:20.424866+0000","last_fullsized":"2024-09-21T01:20:20.424866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:09.647485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968949+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:19:41.968949+0000","last_peered":"2024-09-21T01:19:41.968949+0000","last_clean":"2024-09-21T01:19:41.968949+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:19:41.968949+0000","last_undegraded":"2024-09-21T01:19:41.968949+0000","last_fullsized":"2024-09-21T01:19:41.968949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891745+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:19:41.891745+0000","last_peered":"2024-09-21T01:19:41.891745+0000","last_clean":"2024-09-21T01:19:41.891745+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:19:41.891745+0000","last_undegraded":"2024-09-21T01:19:41.891745+0000","last_fullsized":"2024-09-21T01:19:41.891745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.385873+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:20:04.385873+0000","last_peered":"2024-09-21T01:20:04.385873+0000","last_clean":"2024-09-21T01:20:04.385873+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:20:04.385873+0000","last_undegraded":"2024-09-21T01:20:04.385873+0000","last_fullsized":"2024-09-21T01:20:04.385873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892986+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:19:41.892986+0000","last_peered":"2024-09-21T01:19:41.892986+0000","last_clean":"2024-09-21T01:19:41.892986+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:19:41.892986+0000","last_undegraded":"2024-09-21T01:19:41.892986+0000","last_fullsized":"2024-09-21T01:19:41.892986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891472+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:19:41.891472+0000","last_peered":"2024-09-21T01:19:41.891472+0000","last_clean":"2024-09-21T01:19:41.891472+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:19:41.891472+0000","last_undegraded":"2024-09-21T01:19:41.891472+0000","last_fullsized":"2024-09-21T01:19:41.891472+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.385979+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:04.385979+0000","last_peered":"2024-09-21T01:20:04.385979+0000","last_clean":"2024-09-21T01:20:04.385979+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:04.385979+0000","last_undegraded":"2024-09-21T01:20:04.385979+0000","last_fullsized":"2024-09-21T01:20:04.385979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.897797+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:19:41.897797+0000","last_peered":"2024-09-21T01:19:41.897797+0000","last_clean":"2024-09-21T01:19:41.897797+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:19:41.897797+0000","last_undegraded":"2024-09-21T01:19:41.897797+0000","last_fullsized":"2024-09-21T01:19:41.897797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896004+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:19:41.896004+0000","last_peered":"2024-09-21T01:19:41.896004+0000","last_clean":"2024-09-21T01:19:41.896004+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:19:41.896004+0000","last_undegraded":"2024-09-21T01:19:41.896004+0000","last_fullsized":"2024-09-21T01:19:41.896004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.362702+0000","last_change":"2024-09-21T01:19:03.680497+0000","last_active":"2024-09-21T01:20:03.362702+0000","last_peered":"2024-09-21T01:20:03.362702+0000","last_clean":"2024-09-21T01:20:03.362702+0000","last_became_active":"2024-09-21T01:19:03.679642+0000","last_became_peered":"2024-09-21T01:19:03.679642+0000","last_unstale":"2024-09-21T01:20:03.362702+0000","last_undegraded":"2024-09-21T01:20:03.362702+0000","last_fullsized":"2024-09-21T01:20:03.362702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:25.218921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899464+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:19:41.899464+0000","last_peered":"2024-09-21T01:19:41.899464+0000","last_clean":"2024-09-21T01:19:41.899464+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:19:41.899464+0000","last_undegraded":"2024-09-21T01:19:41.899464+0000","last_fullsized":"2024-09-21T01:19:41.899464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899634+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:19:41.899634+0000","last_peered":"2024-09-21T01:19:41.899634+0000","last_clean":"2024-09-21T01:19:41.899634+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:19:41.899634+0000","last_undegraded":"2024-09-21T01:19:41.899634+0000","last_fullsized":"2024-09-21T01:19:41.899634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.387061+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:04.387061+0000","last_peered":"2024-09-21T01:20:04.387061+0000","last_clean":"2024-09-21T01:20:04.387061+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:04.387061+0000","last_undegraded":"2024-09-21T01:20:04.387061+0000","last_fullsized":"2024-09-21T01:20:04.387061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.178050+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:19:42.178050+0000","last_peered":"2024-09-21T01:19:42.178050+0000","last_clean":"2024-09-21T01:19:42.178050+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:19:42.178050+0000","last_undegraded":"2024-09-21T01:19:42.178050+0000","last_fullsized":"2024-09-21T01:19:42.178050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890513+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:19:41.890513+0000","last_peered":"2024-09-21T01:19:41.890513+0000","last_clean":"2024-09-21T01:19:41.890513+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:19:41.890513+0000","last_undegraded":"2024-09-21T01:19:41.890513+0000","last_fullsized":"2024-09-21T01:19:41.890513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:18.420738+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:20:18.420738+0000","last_peered":"2024-09-21T01:20:18.420738+0000","last_clean":"2024-09-21T01:20:18.420738+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:20:18.420738+0000","last_undegraded":"2024-09-21T01:20:18.420738+0000","last_fullsized":"2024-09-21T01:20:18.420738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.104903+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:19:42.104903+0000","last_peered":"2024-09-21T01:19:42.104903+0000","last_clean":"2024-09-21T01:19:42.104903+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:19:42.104903+0000","last_undegraded":"2024-09-21T01:19:42.104903+0000","last_fullsized":"2024-09-21T01:19:42.104903+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891118+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:19:41.891118+0000","last_peered":"2024-09-21T01:19:41.891118+0000","last_clean":"2024-09-21T01:19:41.891118+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:19:41.891118+0000","last_undegraded":"2024-09-21T01:19:41.891118+0000","last_fullsized":"2024-09-21T01:19:41.891118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386210+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:04.386210+0000","last_peered":"2024-09-21T01:20:04.386210+0000","last_clean":"2024-09-21T01:20:04.386210+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:04.386210+0000","last_undegraded":"2024-09-21T01:20:04.386210+0000","last_fullsized":"2024-09-21T01:20:04.386210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895172+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:19:41.895172+0000","last_peered":"2024-09-21T01:19:41.895172+0000","last_clean":"2024-09-21T01:19:41.895172+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:19:41.895172+0000","last_undegraded":"2024-09-21T01:19:41.895172+0000","last_fullsized":"2024-09-21T01:19:41.895172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177425+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:19:42.177425+0000","last_peered":"2024-09-21T01:19:42.177425+0000","last_clean":"2024-09-21T01:19:42.177425+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:19:42.177425+0000","last_undegraded":"2024-09-21T01:19:42.177425+0000","last_fullsized":"2024-09-21T01:19:42.177425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:05.397006+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:05.397006+0000","last_peered":"2024-09-21T01:20:05.397006+0000","last_clean":"2024-09-21T01:20:05.397006+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:05.397006+0000","last_undegraded":"2024-09-21T01:20:05.397006+0000","last_fullsized":"2024-09-21T01:20:05.397006+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899858+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:19:41.899858+0000","last_peered":"2024-09-21T01:19:41.899858+0000","last_clean":"2024-09-21T01:19:41.899858+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:19:41.899858+0000","last_undegraded":"2024-09-21T01:19:41.899858+0000","last_fullsized":"2024-09-21T01:19:41.899858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890574+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:19:41.890574+0000","last_peered":"2024-09-21T01:19:41.890574+0000","last_clean":"2024-09-21T01:19:41.890574+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:19:41.890574+0000","last_undegraded":"2024-09-21T01:19:41.890574+0000","last_fullsized":"2024-09-21T01:19:41.890574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:19.423370+0000","last_change":"2024-09-21T01:19:03.684959+0000","last_active":"2024-09-21T01:20:19.423370+0000","last_peered":"2024-09-21T01:20:19.423370+0000","last_clean":"2024-09-21T01:20:19.423370+0000","last_became_active":"2024-09-21T01:19:03.684564+0000","last_became_peered":"2024-09-21T01:19:03.684564+0000","last_unstale":"2024-09-21T01:20:19.423370+0000","last_undegraded":"2024-09-21T01:20:19.423370+0000","last_fullsized":"2024-09-21T01:20:19.423370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:15.953025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":73,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898324+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:19:41.898324+0000","last_peered":"2024-09-21T01:19:41.898324+0000","last_clean":"2024-09-21T01:19:41.898324+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:19:41.898324+0000","last_undegraded":"2024-09-21T01:19:41.898324+0000","last_fullsized":"2024-09-21T01:19:41.898324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968951+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:19:41.968951+0000","last_peered":"2024-09-21T01:19:41.968951+0000","last_clean":"2024-09-21T01:19:41.968951+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:19:41.968951+0000","last_undegraded":"2024-09-21T01:19:41.968951+0000","last_fullsized":"2024-09-21T01:19:41.968951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.381440+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:03.381440+0000","last_peered":"2024-09-21T01:20:03.381440+0000","last_clean":"2024-09-21T01:20:03.381440+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:03.381440+0000","last_undegraded":"2024-09-21T01:20:03.381440+0000","last_fullsized":"2024-09-21T01:20:03.381440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893899+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:19:41.893899+0000","last_peered":"2024-09-21T01:19:41.893899+0000","last_clean":"2024-09-21T01:19:41.893899+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:19:41.893899+0000","last_undegraded":"2024-09-21T01:19:41.893899+0000","last_fullsized":"2024-09-21T01:19:41.893899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893355+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:19:41.893355+0000","last_peered":"2024-09-21T01:19:41.893355+0000","last_clean":"2024-09-21T01:19:41.893355+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:19:41.893355+0000","last_undegraded":"2024-09-21T01:19:41.893355+0000","last_fullsized":"2024-09-21T01:19:41.893355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.385149+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:04.385149+0000","last_peered":"2024-09-21T01:20:04.385149+0000","last_clean":"2024-09-21T01:20:04.385149+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:04.385149+0000","last_undegraded":"2024-09-21T01:20:04.385149+0000","last_fullsized":"2024-09-21T01:20:04.385149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896845+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:19:41.896845+0000","last_peered":"2024-09-21T01:19:41.896845+0000","last_clean":"2024-09-21T01:19:41.896845+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:19:41.896845+0000","last_undegraded":"2024-09-21T01:19:41.896845+0000","last_fullsized":"2024-09-21T01:19:41.896845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.969655+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:19:41.969655+0000","last_peered":"2024-09-21T01:19:41.969655+0000","last_clean":"2024-09-21T01:19:41.969655+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:19:41.969655+0000","last_undegraded":"2024-09-21T01:19:41.969655+0000","last_fullsized":"2024-09-21T01:19:41.969655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:19.422280+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:20:19.422280+0000","last_peered":"2024-09-21T01:20:19.422280+0000","last_clean":"2024-09-21T01:20:19.422280+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:20:19.422280+0000","last_undegraded":"2024-09-21T01:20:19.422280+0000","last_fullsized":"2024-09-21T01:20:19.422280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.102796+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:19:42.102796+0000","last_peered":"2024-09-21T01:19:42.102796+0000","last_clean":"2024-09-21T01:19:42.102796+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:19:42.102796+0000","last_undegraded":"2024-09-21T01:19:42.102796+0000","last_fullsized":"2024-09-21T01:19:42.102796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892829+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:19:41.892829+0000","last_peered":"2024-09-21T01:19:41.892829+0000","last_clean":"2024-09-21T01:19:41.892829+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:19:41.892829+0000","last_undegraded":"2024-09-21T01:19:41.892829+0000","last_fullsized":"2024-09-21T01:19:41.892829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.897128+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:19:41.897128+0000","last_peered":"2024-09-21T01:19:41.897128+0000","last_clean":"2024-09-21T01:19:41.897128+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:19:41.897128+0000","last_undegraded":"2024-09-21T01:19:41.897128+0000","last_fullsized":"2024-09-21T01:19:41.897128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386310+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:04.386310+0000","last_peered":"2024-09-21T01:20:04.386310+0000","last_clean":"2024-09-21T01:20:04.386310+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:04.386310+0000","last_undegraded":"2024-09-21T01:20:04.386310+0000","last_fullsized":"2024-09-21T01:20:04.386310+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899821+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:19:41.899821+0000","last_peered":"2024-09-21T01:19:41.899821+0000","last_clean":"2024-09-21T01:19:41.899821+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:19:41.899821+0000","last_undegraded":"2024-09-21T01:19:41.899821+0000","last_fullsized":"2024-09-21T01:19:41.899821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103631+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:19:42.103631+0000","last_peered":"2024-09-21T01:19:42.103631+0000","last_clean":"2024-09-21T01:19:42.103631+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:19:42.103631+0000","last_undegraded":"2024-09-21T01:19:42.103631+0000","last_fullsized":"2024-09-21T01:19:42.103631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:02.372616+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:02.372616+0000","last_peered":"2024-09-21T01:20:02.372616+0000","last_clean":"2024-09-21T01:20:02.372616+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:02.372616+0000","last_undegraded":"2024-09-21T01:20:02.372616+0000","last_fullsized":"2024-09-21T01:20:02.372616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.380740+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:03.380740+0000","last_peered":"2024-09-21T01:20:03.380740+0000","last_clean":"2024-09-21T01:20:03.380740+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:03.380740+0000","last_undegraded":"2024-09-21T01:20:03.380740+0000","last_fullsized":"2024-09-21T01:20:03.380740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899457+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:19:41.899457+0000","last_peered":"2024-09-21T01:19:41.899457+0000","last_clean":"2024-09-21T01:19:41.899457+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:19:41.899457+0000","last_undegraded":"2024-09-21T01:19:41.899457+0000","last_fullsized":"2024-09-21T01:19:41.899457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895505+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:19:41.895505+0000","last_peered":"2024-09-21T01:19:41.895505+0000","last_clean":"2024-09-21T01:19:41.895505+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:19:41.895505+0000","last_undegraded":"2024-09-21T01:19:41.895505+0000","last_fullsized":"2024-09-21T01:19:41.895505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:20.424998+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:20:20.424998+0000","last_peered":"2024-09-21T01:20:20.424998+0000","last_clean":"2024-09-21T01:20:20.424998+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:20:20.424998+0000","last_undegraded":"2024-09-21T01:20:20.424998+0000","last_fullsized":"2024-09-21T01:20:20.424998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890353+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:19:41.890353+0000","last_peered":"2024-09-21T01:19:41.890353+0000","last_clean":"2024-09-21T01:19:41.890353+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:19:41.890353+0000","last_undegraded":"2024-09-21T01:19:41.890353+0000","last_fullsized":"2024-09-21T01:19:41.890353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895959+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:19:41.895959+0000","last_peered":"2024-09-21T01:19:41.895959+0000","last_clean":"2024-09-21T01:19:41.895959+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:19:41.895959+0000","last_undegraded":"2024-09-21T01:19:41.895959+0000","last_fullsized":"2024-09-21T01:19:41.895959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.894868+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:19:41.894868+0000","last_peered":"2024-09-21T01:19:41.894868+0000","last_clean":"2024-09-21T01:19:41.894868+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:19:41.894868+0000","last_undegraded":"2024-09-21T01:19:41.894868+0000","last_fullsized":"2024-09-21T01:19:41.894868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.902359+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:19:41.902359+0000","last_peered":"2024-09-21T01:19:41.902359+0000","last_clean":"2024-09-21T01:19:41.902359+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:19:41.902359+0000","last_undegraded":"2024-09-21T01:19:41.902359+0000","last_fullsized":"2024-09-21T01:19:41.902359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895565+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:19:41.895565+0000","last_peered":"2024-09-21T01:19:41.895565+0000","last_clean":"2024-09-21T01:19:41.895565+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:19:41.895565+0000","last_undegraded":"2024-09-21T01:19:41.895565+0000","last_fullsized":"2024-09-21T01:19:41.895565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893289+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:19:41.893289+0000","last_peered":"2024-09-21T01:19:41.893289+0000","last_clean":"2024-09-21T01:19:41.893289+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:19:41.893289+0000","last_undegraded":"2024-09-21T01:19:41.893289+0000","last_fullsized":"2024-09-21T01:19:41.893289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177820+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:19:42.177820+0000","last_peered":"2024-09-21T01:19:42.177820+0000","last_clean":"2024-09-21T01:19:42.177820+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:19:42.177820+0000","last_undegraded":"2024-09-21T01:19:42.177820+0000","last_fullsized":"2024-09-21T01:19:42.177820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896146+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:19:41.896146+0000","last_peered":"2024-09-21T01:19:41.896146+0000","last_clean":"2024-09-21T01:19:41.896146+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:19:41.896146+0000","last_undegraded":"2024-09-21T01:19:41.896146+0000","last_fullsized":"2024-09-21T01:19:41.896146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896250+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:19:41.896250+0000","last_peered":"2024-09-21T01:19:41.896250+0000","last_clean":"2024-09-21T01:19:41.896250+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:19:41.896250+0000","last_undegraded":"2024-09-21T01:19:41.896250+0000","last_fullsized":"2024-09-21T01:19:41.896250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103688+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:19:42.103688+0000","last_peered":"2024-09-21T01:19:42.103688+0000","last_clean":"2024-09-21T01:19:42.103688+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:19:42.103688+0000","last_undegraded":"2024-09-21T01:19:42.103688+0000","last_fullsized":"2024-09-21T01:19:42.103688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898015+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:19:41.898015+0000","last_peered":"2024-09-21T01:19:41.898015+0000","last_clean":"2024-09-21T01:19:41.898015+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:19:41.898015+0000","last_undegraded":"2024-09-21T01:19:41.898015+0000","last_fullsized":"2024-09-21T01:19:41.898015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103042+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:19:42.103042+0000","last_peered":"2024-09-21T01:19:42.103042+0000","last_clean":"2024-09-21T01:19:42.103042+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:19:42.103042+0000","last_undegraded":"2024-09-21T01:19:42.103042+0000","last_fullsized":"2024-09-21T01:19:42.103042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.969128+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:19:41.969128+0000","last_peered":"2024-09-21T01:19:41.969128+0000","last_clean":"2024-09-21T01:19:41.969128+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:19:41.969128+0000","last_undegraded":"2024-09-21T01:19:41.969128+0000","last_fullsized":"2024-09-21T01:19:41.969128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177796+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:19:42.177796+0000","last_peered":"2024-09-21T01:19:42.177796+0000","last_clean":"2024-09-21T01:19:42.177796+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:19:42.177796+0000","last_undegraded":"2024-09-21T01:19:42.177796+0000","last_fullsized":"2024-09-21T01:19:42.177796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895308+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:19:41.895308+0000","last_peered":"2024-09-21T01:19:41.895308+0000","last_clean":"2024-09-21T01:19:41.895308+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:19:41.895308+0000","last_undegraded":"2024-09-21T01:19:41.895308+0000","last_fullsized":"2024-09-21T01:19:41.895308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896367+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:19:41.896367+0000","last_peered":"2024-09-21T01:19:41.896367+0000","last_clean":"2024-09-21T01:19:41.896367+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:19:41.896367+0000","last_undegraded":"2024-09-21T01:19:41.896367+0000","last_fullsized":"2024-09-21T01:19:41.896367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898594+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:19:41.898594+0000","last_peered":"2024-09-21T01:19:41.898594+0000","last_clean":"2024-09-21T01:19:41.898594+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:19:41.898594+0000","last_undegraded":"2024-09-21T01:19:41.898594+0000","last_fullsized":"2024-09-21T01:19:41.898594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896653+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:19:41.896653+0000","last_peered":"2024-09-21T01:19:41.896653+0000","last_clean":"2024-09-21T01:19:41.896653+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:19:41.896653+0000","last_undegraded":"2024-09-21T01:19:41.896653+0000","last_fullsized":"2024-09-21T01:19:41.896653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968521+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:19:41.968521+0000","last_peered":"2024-09-21T01:19:41.968521+0000","last_clean":"2024-09-21T01:19:41.968521+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:19:41.968521+0000","last_undegraded":"2024-09-21T01:19:41.968521+0000","last_fullsized":"2024-09-21T01:19:41.968521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892640+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:19:41.892640+0000","last_peered":"2024-09-21T01:19:41.892640+0000","last_clean":"2024-09-21T01:19:41.892640+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:19:41.892640+0000","last_undegraded":"2024-09-21T01:19:41.892640+0000","last_fullsized":"2024-09-21T01:19:41.892640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892851+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:19:41.892851+0000","last_peered":"2024-09-21T01:19:41.892851+0000","last_clean":"2024-09-21T01:19:41.892851+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:19:41.892851+0000","last_undegraded":"2024-09-21T01:19:41.892851+0000","last_fullsized":"2024-09-21T01:19:41.892851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177726+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:19:42.177726+0000","last_peered":"2024-09-21T01:19:42.177726+0000","last_clean":"2024-09-21T01:19:42.177726+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:19:42.177726+0000","last_undegraded":"2024-09-21T01:19:42.177726+0000","last_fullsized":"2024-09-21T01:19:42.177726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893877+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:19:41.893877+0000","last_peered":"2024-09-21T01:19:41.893877+0000","last_clean":"2024-09-21T01:19:41.893877+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:19:41.893877+0000","last_undegraded":"2024-09-21T01:19:41.893877+0000","last_fullsized":"2024-09-21T01:19:41.893877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891364+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:19:41.891364+0000","last_peered":"2024-09-21T01:19:41.891364+0000","last_clean":"2024-09-21T01:19:41.891364+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:19:41.891364+0000","last_undegraded":"2024-09-21T01:19:41.891364+0000","last_fullsized":"2024-09-21T01:19:41.891364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896628+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:19:41.896628+0000","last_peered":"2024-09-21T01:19:41.896628+0000","last_clean":"2024-09-21T01:19:41.896628+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:19:41.896628+0000","last_undegraded":"2024-09-21T01:19:41.896628+0000","last_fullsized":"2024-09-21T01:19:41.896628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.969655+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:19:41.969655+0000","last_peered":"2024-09-21T01:19:41.969655+0000","last_clean":"2024-09-21T01:19:41.969655+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:19:41.969655+0000","last_undegraded":"2024-09-21T01:19:41.969655+0000","last_fullsized":"2024-09-21T01:19:41.969655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.102966+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:19:42.102966+0000","last_peered":"2024-09-21T01:19:42.102966+0000","last_clean":"2024-09-21T01:19:42.102966+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:19:42.102966+0000","last_undegraded":"2024-09-21T01:19:42.102966+0000","last_fullsized":"2024-09-21T01:19:42.102966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890676+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:19:41.890676+0000","last_peered":"2024-09-21T01:19:41.890676+0000","last_clean":"2024-09-21T01:19:41.890676+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:19:41.890676+0000","last_undegraded":"2024-09-21T01:19:41.890676+0000","last_fullsized":"2024-09-21T01:19:41.890676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895930+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:19:41.895930+0000","last_peered":"2024-09-21T01:19:41.895930+0000","last_clean":"2024-09-21T01:19:41.895930+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:19:41.895930+0000","last_undegraded":"2024-09-21T01:19:41.895930+0000","last_fullsized":"2024-09-21T01:19:41.895930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890219+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:19:41.890219+0000","last_peered":"2024-09-21T01:19:41.890219+0000","last_clean":"2024-09-21T01:19:41.890219+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:19:41.890219+0000","last_undegraded":"2024-09-21T01:19:41.890219+0000","last_fullsized":"2024-09-21T01:19:41.890219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893753+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:19:41.893753+0000","last_peered":"2024-09-21T01:19:41.893753+0000","last_clean":"2024-09-21T01:19:41.893753+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:19:41.893753+0000","last_undegraded":"2024-09-21T01:19:41.893753+0000","last_fullsized":"2024-09-21T01:19:41.893753+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893340+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:19:41.893340+0000","last_peered":"2024-09-21T01:19:41.893340+0000","last_clean":"2024-09-21T01:19:41.893340+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:19:41.893340+0000","last_undegraded":"2024-09-21T01:19:41.893340+0000","last_fullsized":"2024-09-21T01:19:41.893340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892982+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:19:41.892982+0000","last_peered":"2024-09-21T01:19:41.892982+0000","last_clean":"2024-09-21T01:19:41.892982+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:19:41.892982+0000","last_undegraded":"2024-09-21T01:19:41.892982+0000","last_fullsized":"2024-09-21T01:19:41.892982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896526+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:19:41.896526+0000","last_peered":"2024-09-21T01:19:41.896526+0000","last_clean":"2024-09-21T01:19:41.896526+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:19:41.896526+0000","last_undegraded":"2024-09-21T01:19:41.896526+0000","last_fullsized":"2024-09-21T01:19:41.896526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63748,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":201,"num_read_kb":173,"num_write":244,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":191244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":516031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874274,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1708032,"data_stored":1163878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037800,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":515961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":516339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430259,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1113227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":511465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1108418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":515930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":511173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":569650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542229,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":569039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705755,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":512921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:20:26.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:20:26.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:26 smithi098 ceph-mon[34372]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:20:26.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:26 smithi027 ceph-mon[30375]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:20:26.936 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:26.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:26 smithi089 ceph-mon[34486]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:20:27.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 -- 172.21.15.27:0/2918192813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 msgr2=0x7fd87c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:27.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 --2- 172.21.15.27:0/2918192813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c112840 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd874009ca0 tx=0x7fd87402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 -- 172.21.15.27:0/2918192813 shutdown_connections 2024-09-21T01:20:27.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 --2- 172.21.15.27:0/2918192813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c112840 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 --2- 172.21.15.27:0/2918192813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 --2- 172.21.15.27:0/2918192813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd87c105d80 0x7fd87c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 -- 172.21.15.27:0/2918192813 >> 172.21.15.27:0/2918192813 conn(0x7fd87c101810 msgr2=0x7fd87c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:27.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.591+0000 7fd881e00640 1 -- 172.21.15.27:0/2918192813 shutdown_connections 2024-09-21T01:20:27.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.592+0000 7fd881e00640 1 -- 172.21.15.27:0/2918192813 wait complete. 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 Processor -- start 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 -- start start 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd87c105d80 0x7fd87c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c115cd0 con 0x7fd87c10c010 2024-09-21T01:20:27.594 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c115b50 con 0x7fd87c10b6c0 2024-09-21T01:20:27.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd881e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c115e50 con 0x7fd87c105d80 2024-09-21T01:20:27.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd87aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd87c105d80 0x7fd87c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:27.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.593+0000 7fd873e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:27.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58848/0 (socket says 172.21.15.27:58848) 2024-09-21T01:20:27.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 -- 172.21.15.27:0/2968760444 learned_addr learned my addr 172.21.15.27:0/2968760444 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:27.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd87b400640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 -- 172.21.15.27:0/2968760444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd87c105d80 msgr2=0x7fd87c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd87c105d80 0x7fd87c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 -- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 msgr2=0x7fd87c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 -- 172.21.15.27:0/2968760444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c1a25d0 con 0x7fd87c10b6c0 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd87b400640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.594+0000 7fd873e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c19cc00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd86400e9f0 tx=0x7fd86400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.595+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86400ccc0 con 0x7fd87c10b6c0 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.595+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1a28c0 con 0x7fd87c10b6c0 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.595+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8640108e0 con 0x7fd87c10b6c0 2024-09-21T01:20:27.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.595+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864018860 con 0x7fd87c10b6c0 2024-09-21T01:20:27.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.595+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1a9020 con 0x7fd87c10b6c0 2024-09-21T01:20:27.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.596+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c106190 con 0x7fd87c10b6c0 2024-09-21T01:20:27.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.598+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd864010430 con 0x7fd87c10b6c0 2024-09-21T01:20:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.600+0000 7fd880a00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd8600b65f0 0x7fd8600b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.600+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fd8640e8140 con 0x7fd87c10b6c0 2024-09-21T01:20:27.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.600+0000 7fd87aa00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd8600b65f0 0x7fd8600b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:27.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.601+0000 7fd87aa00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd8600b65f0 0x7fd8600b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009bd0 tx=0x7fd86c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:27.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.601+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd8640ae3f0 con 0x7fd87c10b6c0 2024-09-21T01:20:27.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:27 smithi027 ceph-mon[30375]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-21T01:20:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.796+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd87c063800 con 0x7fd8600b65f0 2024-09-21T01:20:27.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.803+0000 7fd880a00640 1 -- 172.21.15.27:0/2968760444 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340389 (secure 0 0 0) 0x7fd87c063800 con 0x7fd8600b65f0 2024-09-21T01:20:27.807 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:27.809 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.808+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd8600b65f0 msgr2=0x7fd8600b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.808+0000 7fd881e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd8600b65f0 0x7fd8600b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009bd0 tx=0x7fd86c005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.808+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 msgr2=0x7fd87c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.808+0000 7fd881e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c19cc00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd86400e9f0 tx=0x7fd86400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.808+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 shutdown_connections 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd8600b65f0 0x7fd8600b8b00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87c10c010 0x7fd87c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd87c10b6c0 0x7fd87c19cc00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 --2- 172.21.15.27:0/2968760444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd87c105d80 0x7fd87c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 >> 172.21.15.27:0/2968760444 conn(0x7fd87c101810 msgr2=0x7fd87c108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 shutdown_connections 2024-09-21T01:20:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:27.809+0000 7fd881e00640 1 -- 172.21.15.27:0/2968760444 wait complete. 2024-09-21T01:20:27.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:27 smithi098 ceph-mon[34372]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-21T01:20:28.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:27 smithi089 ceph-mon[34486]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-21T01:20:28.709 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-21T01:20:26.438400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658615,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":313,"num_read_kb":262,"num_write":370,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371596,"kb_used_data":14028,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570228,"statfs":{"total":1151940427776,"available":1151559913472,"internally_reserved":0,"allocated":14364672,"data_stored":8128297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57592,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":124,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003927"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:22.426474+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:20:22.426474+0000","last_peered":"2024-09-21T01:20:22.426474+0000","last_clean":"2024-09-21T01:20:22.426474+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:20:22.426474+0000","last_undegraded":"2024-09-21T01:20:22.426474+0000","last_fullsized":"2024-09-21T01:20:22.426474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.902018+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:19:41.902018+0000","last_peered":"2024-09-21T01:19:41.902018+0000","last_clean":"2024-09-21T01:19:41.902018+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:19:41.902018+0000","last_undegraded":"2024-09-21T01:19:41.902018+0000","last_fullsized":"2024-09-21T01:19:41.902018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896721+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:19:41.896721+0000","last_peered":"2024-09-21T01:19:41.896721+0000","last_clean":"2024-09-21T01:19:41.896721+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:19:41.896721+0000","last_undegraded":"2024-09-21T01:19:41.896721+0000","last_fullsized":"2024-09-21T01:19:41.896721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.104024+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:19:42.104024+0000","last_peered":"2024-09-21T01:19:42.104024+0000","last_clean":"2024-09-21T01:19:42.104024+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:19:42.104024+0000","last_undegraded":"2024-09-21T01:19:42.104024+0000","last_fullsized":"2024-09-21T01:19:42.104024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893193+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:19:41.893193+0000","last_peered":"2024-09-21T01:19:41.893193+0000","last_clean":"2024-09-21T01:19:41.893193+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:19:41.893193+0000","last_undegraded":"2024-09-21T01:19:41.893193+0000","last_fullsized":"2024-09-21T01:19:41.893193+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892871+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:19:41.892871+0000","last_peered":"2024-09-21T01:19:41.892871+0000","last_clean":"2024-09-21T01:19:41.892871+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:19:41.892871+0000","last_undegraded":"2024-09-21T01:19:41.892871+0000","last_fullsized":"2024-09-21T01:19:41.892871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:05.397635+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:20:05.397635+0000","last_peered":"2024-09-21T01:20:05.397635+0000","last_clean":"2024-09-21T01:20:05.397635+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:20:05.397635+0000","last_undegraded":"2024-09-21T01:20:05.397635+0000","last_fullsized":"2024-09-21T01:20:05.397635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968924+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:19:41.968924+0000","last_peered":"2024-09-21T01:19:41.968924+0000","last_clean":"2024-09-21T01:19:41.968924+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:19:41.968924+0000","last_undegraded":"2024-09-21T01:19:41.968924+0000","last_fullsized":"2024-09-21T01:19:41.968924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893328+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:19:41.893328+0000","last_peered":"2024-09-21T01:19:41.893328+0000","last_clean":"2024-09-21T01:19:41.893328+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:19:41.893328+0000","last_undegraded":"2024-09-21T01:19:41.893328+0000","last_fullsized":"2024-09-21T01:19:41.893328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386940+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:04.386940+0000","last_peered":"2024-09-21T01:20:04.386940+0000","last_clean":"2024-09-21T01:20:04.386940+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:04.386940+0000","last_undegraded":"2024-09-21T01:20:04.386940+0000","last_fullsized":"2024-09-21T01:20:04.386940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.179169+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:19:42.179169+0000","last_peered":"2024-09-21T01:19:42.179169+0000","last_clean":"2024-09-21T01:19:42.179169+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:19:42.179169+0000","last_undegraded":"2024-09-21T01:19:42.179169+0000","last_fullsized":"2024-09-21T01:19:42.179169+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892994+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:19:41.892994+0000","last_peered":"2024-09-21T01:19:41.892994+0000","last_clean":"2024-09-21T01:19:41.892994+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:19:41.892994+0000","last_undegraded":"2024-09-21T01:19:41.892994+0000","last_fullsized":"2024-09-21T01:19:41.892994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890856+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:19:41.890856+0000","last_peered":"2024-09-21T01:19:41.890856+0000","last_clean":"2024-09-21T01:19:41.890856+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:19:41.890856+0000","last_undegraded":"2024-09-21T01:19:41.890856+0000","last_fullsized":"2024-09-21T01:19:41.890856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:01.359747+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:01.359747+0000","last_peered":"2024-09-21T01:20:01.359747+0000","last_clean":"2024-09-21T01:20:01.359747+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:01.359747+0000","last_undegraded":"2024-09-21T01:20:01.359747+0000","last_fullsized":"2024-09-21T01:20:01.359747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893205+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:19:41.893205+0000","last_peered":"2024-09-21T01:19:41.893205+0000","last_clean":"2024-09-21T01:19:41.893205+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:19:41.893205+0000","last_undegraded":"2024-09-21T01:19:41.893205+0000","last_fullsized":"2024-09-21T01:19:41.893205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103086+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:19:42.103086+0000","last_peered":"2024-09-21T01:19:42.103086+0000","last_clean":"2024-09-21T01:19:42.103086+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:19:42.103086+0000","last_undegraded":"2024-09-21T01:19:42.103086+0000","last_fullsized":"2024-09-21T01:19:42.103086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.379872+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:03.379872+0000","last_peered":"2024-09-21T01:20:03.379872+0000","last_clean":"2024-09-21T01:20:03.379872+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:03.379872+0000","last_undegraded":"2024-09-21T01:20:03.379872+0000","last_fullsized":"2024-09-21T01:20:03.379872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898484+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:19:41.898484+0000","last_peered":"2024-09-21T01:19:41.898484+0000","last_clean":"2024-09-21T01:19:41.898484+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:19:41.898484+0000","last_undegraded":"2024-09-21T01:19:41.898484+0000","last_fullsized":"2024-09-21T01:19:41.898484+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.894872+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:19:41.894872+0000","last_peered":"2024-09-21T01:19:41.894872+0000","last_clean":"2024-09-21T01:19:41.894872+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:19:41.894872+0000","last_undegraded":"2024-09-21T01:19:41.894872+0000","last_fullsized":"2024-09-21T01:19:41.894872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386175+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:04.386175+0000","last_peered":"2024-09-21T01:20:04.386175+0000","last_clean":"2024-09-21T01:20:04.386175+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:04.386175+0000","last_undegraded":"2024-09-21T01:20:04.386175+0000","last_fullsized":"2024-09-21T01:20:04.386175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898591+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:19:41.898591+0000","last_peered":"2024-09-21T01:19:41.898591+0000","last_clean":"2024-09-21T01:19:41.898591+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:19:41.898591+0000","last_undegraded":"2024-09-21T01:19:41.898591+0000","last_fullsized":"2024-09-21T01:19:41.898591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890147+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:19:41.890147+0000","last_peered":"2024-09-21T01:19:41.890147+0000","last_clean":"2024-09-21T01:19:41.890147+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:19:41.890147+0000","last_undegraded":"2024-09-21T01:19:41.890147+0000","last_fullsized":"2024-09-21T01:19:41.890147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:01.360664+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:01.360664+0000","last_peered":"2024-09-21T01:20:01.360664+0000","last_clean":"2024-09-21T01:20:01.360664+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:01.360664+0000","last_undegraded":"2024-09-21T01:20:01.360664+0000","last_fullsized":"2024-09-21T01:20:01.360664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893346+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:19:41.893346+0000","last_peered":"2024-09-21T01:19:41.893346+0000","last_clean":"2024-09-21T01:19:41.893346+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:19:41.893346+0000","last_undegraded":"2024-09-21T01:19:41.893346+0000","last_fullsized":"2024-09-21T01:19:41.893346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893111+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:19:41.893111+0000","last_peered":"2024-09-21T01:19:41.893111+0000","last_clean":"2024-09-21T01:19:41.893111+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:19:41.893111+0000","last_undegraded":"2024-09-21T01:19:41.893111+0000","last_fullsized":"2024-09-21T01:19:41.893111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.387246+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:04.387246+0000","last_peered":"2024-09-21T01:20:04.387246+0000","last_clean":"2024-09-21T01:20:04.387246+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:04.387246+0000","last_undegraded":"2024-09-21T01:20:04.387246+0000","last_fullsized":"2024-09-21T01:20:04.387246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896923+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:19:41.896923+0000","last_peered":"2024-09-21T01:19:41.896923+0000","last_clean":"2024-09-21T01:19:41.896923+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:19:41.896923+0000","last_undegraded":"2024-09-21T01:19:41.896923+0000","last_fullsized":"2024-09-21T01:19:41.896923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103705+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:19:42.103705+0000","last_peered":"2024-09-21T01:19:42.103705+0000","last_clean":"2024-09-21T01:19:42.103705+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:19:42.103705+0000","last_undegraded":"2024-09-21T01:19:42.103705+0000","last_fullsized":"2024-09-21T01:19:42.103705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:20.747971+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:20:20.747971+0000","last_peered":"2024-09-21T01:20:20.747971+0000","last_clean":"2024-09-21T01:20:20.747971+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:20:20.747971+0000","last_undegraded":"2024-09-21T01:20:20.747971+0000","last_fullsized":"2024-09-21T01:20:20.747971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.102599+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:19:42.102599+0000","last_peered":"2024-09-21T01:19:42.102599+0000","last_clean":"2024-09-21T01:19:42.102599+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:19:42.102599+0000","last_undegraded":"2024-09-21T01:19:42.102599+0000","last_fullsized":"2024-09-21T01:19:42.102599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177889+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:19:42.177889+0000","last_peered":"2024-09-21T01:19:42.177889+0000","last_clean":"2024-09-21T01:19:42.177889+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:19:42.177889+0000","last_undegraded":"2024-09-21T01:19:42.177889+0000","last_fullsized":"2024-09-21T01:19:42.177889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.387739+0000","last_change":"2024-09-21T01:19:03.692864+0000","last_active":"2024-09-21T01:20:04.387739+0000","last_peered":"2024-09-21T01:20:04.387739+0000","last_clean":"2024-09-21T01:20:04.387739+0000","last_became_active":"2024-09-21T01:19:03.692585+0000","last_became_peered":"2024-09-21T01:19:03.692585+0000","last_unstale":"2024-09-21T01:20:04.387739+0000","last_undegraded":"2024-09-21T01:20:04.387739+0000","last_fullsized":"2024-09-21T01:20:04.387739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:01:53.429693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895152+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:19:41.895152+0000","last_peered":"2024-09-21T01:19:41.895152+0000","last_clean":"2024-09-21T01:19:41.895152+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:19:41.895152+0000","last_undegraded":"2024-09-21T01:19:41.895152+0000","last_fullsized":"2024-09-21T01:19:41.895152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896304+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:19:41.896304+0000","last_peered":"2024-09-21T01:19:41.896304+0000","last_clean":"2024-09-21T01:19:41.896304+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:19:41.896304+0000","last_undegraded":"2024-09-21T01:19:41.896304+0000","last_fullsized":"2024-09-21T01:19:41.896304+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899130+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:19:41.899130+0000","last_peered":"2024-09-21T01:19:41.899130+0000","last_clean":"2024-09-21T01:19:41.899130+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:19:41.899130+0000","last_undegraded":"2024-09-21T01:19:41.899130+0000","last_fullsized":"2024-09-21T01:19:41.899130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890147+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:19:41.890147+0000","last_peered":"2024-09-21T01:19:41.890147+0000","last_clean":"2024-09-21T01:19:41.890147+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:19:41.890147+0000","last_undegraded":"2024-09-21T01:19:41.890147+0000","last_fullsized":"2024-09-21T01:19:41.890147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898803+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:19:41.898803+0000","last_peered":"2024-09-21T01:19:41.898803+0000","last_clean":"2024-09-21T01:19:41.898803+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:19:41.898803+0000","last_undegraded":"2024-09-21T01:19:41.898803+0000","last_fullsized":"2024-09-21T01:19:41.898803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:23.428831+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:20:23.428831+0000","last_peered":"2024-09-21T01:20:23.428831+0000","last_clean":"2024-09-21T01:20:23.428831+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:20:23.428831+0000","last_undegraded":"2024-09-21T01:20:23.428831+0000","last_fullsized":"2024-09-21T01:20:23.428831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177611+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:19:42.177611+0000","last_peered":"2024-09-21T01:19:42.177611+0000","last_clean":"2024-09-21T01:19:42.177611+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:19:42.177611+0000","last_undegraded":"2024-09-21T01:19:42.177611+0000","last_fullsized":"2024-09-21T01:19:42.177611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.900102+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:19:41.900102+0000","last_peered":"2024-09-21T01:19:41.900102+0000","last_clean":"2024-09-21T01:19:41.900102+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:19:41.900102+0000","last_undegraded":"2024-09-21T01:19:41.900102+0000","last_fullsized":"2024-09-21T01:19:41.900102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:22.426436+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:20:22.426436+0000","last_peered":"2024-09-21T01:20:22.426436+0000","last_clean":"2024-09-21T01:20:22.426436+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:20:22.426436+0000","last_undegraded":"2024-09-21T01:20:22.426436+0000","last_fullsized":"2024-09-21T01:20:22.426436+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.178078+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:19:42.178078+0000","last_peered":"2024-09-21T01:19:42.178078+0000","last_clean":"2024-09-21T01:19:42.178078+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:19:42.178078+0000","last_undegraded":"2024-09-21T01:19:42.178078+0000","last_fullsized":"2024-09-21T01:19:42.178078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177246+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:19:42.177246+0000","last_peered":"2024-09-21T01:19:42.177246+0000","last_clean":"2024-09-21T01:19:42.177246+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:19:42.177246+0000","last_undegraded":"2024-09-21T01:19:42.177246+0000","last_fullsized":"2024-09-21T01:19:42.177246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:02.368655+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:02.368655+0000","last_peered":"2024-09-21T01:20:02.368655+0000","last_clean":"2024-09-21T01:20:02.368655+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:02.368655+0000","last_undegraded":"2024-09-21T01:20:02.368655+0000","last_fullsized":"2024-09-21T01:20:02.368655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.180220+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:19:42.180220+0000","last_peered":"2024-09-21T01:19:42.180220+0000","last_clean":"2024-09-21T01:19:42.180220+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:19:42.180220+0000","last_undegraded":"2024-09-21T01:19:42.180220+0000","last_fullsized":"2024-09-21T01:19:42.180220+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.971047+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:19:41.971047+0000","last_peered":"2024-09-21T01:19:41.971047+0000","last_clean":"2024-09-21T01:19:41.971047+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:19:41.971047+0000","last_undegraded":"2024-09-21T01:19:41.971047+0000","last_fullsized":"2024-09-21T01:19:41.971047+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:05.396996+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:05.396996+0000","last_peered":"2024-09-21T01:20:05.396996+0000","last_clean":"2024-09-21T01:20:05.396996+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:05.396996+0000","last_undegraded":"2024-09-21T01:20:05.396996+0000","last_fullsized":"2024-09-21T01:20:05.396996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890786+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:19:41.890786+0000","last_peered":"2024-09-21T01:19:41.890786+0000","last_clean":"2024-09-21T01:19:41.890786+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:19:41.890786+0000","last_undegraded":"2024-09-21T01:19:41.890786+0000","last_fullsized":"2024-09-21T01:19:41.890786+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891338+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:19:41.891338+0000","last_peered":"2024-09-21T01:19:41.891338+0000","last_clean":"2024-09-21T01:19:41.891338+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:19:41.891338+0000","last_undegraded":"2024-09-21T01:19:41.891338+0000","last_fullsized":"2024-09-21T01:19:41.891338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891420+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:19:41.891420+0000","last_peered":"2024-09-21T01:19:41.891420+0000","last_clean":"2024-09-21T01:19:41.891420+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:19:41.891420+0000","last_undegraded":"2024-09-21T01:19:41.891420+0000","last_fullsized":"2024-09-21T01:19:41.891420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:20.424866+0000","last_change":"2024-09-21T01:19:04.020705+0000","last_active":"2024-09-21T01:20:20.424866+0000","last_peered":"2024-09-21T01:20:20.424866+0000","last_clean":"2024-09-21T01:20:20.424866+0000","last_became_active":"2024-09-21T01:19:04.020234+0000","last_became_peered":"2024-09-21T01:19:04.020234+0000","last_unstale":"2024-09-21T01:20:20.424866+0000","last_undegraded":"2024-09-21T01:20:20.424866+0000","last_fullsized":"2024-09-21T01:20:20.424866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:09.647485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968949+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:19:41.968949+0000","last_peered":"2024-09-21T01:19:41.968949+0000","last_clean":"2024-09-21T01:19:41.968949+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:19:41.968949+0000","last_undegraded":"2024-09-21T01:19:41.968949+0000","last_fullsized":"2024-09-21T01:19:41.968949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891745+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:19:41.891745+0000","last_peered":"2024-09-21T01:19:41.891745+0000","last_clean":"2024-09-21T01:19:41.891745+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:19:41.891745+0000","last_undegraded":"2024-09-21T01:19:41.891745+0000","last_fullsized":"2024-09-21T01:19:41.891745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.385873+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:20:04.385873+0000","last_peered":"2024-09-21T01:20:04.385873+0000","last_clean":"2024-09-21T01:20:04.385873+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:20:04.385873+0000","last_undegraded":"2024-09-21T01:20:04.385873+0000","last_fullsized":"2024-09-21T01:20:04.385873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892986+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:19:41.892986+0000","last_peered":"2024-09-21T01:19:41.892986+0000","last_clean":"2024-09-21T01:19:41.892986+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:19:41.892986+0000","last_undegraded":"2024-09-21T01:19:41.892986+0000","last_fullsized":"2024-09-21T01:19:41.892986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891472+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:19:41.891472+0000","last_peered":"2024-09-21T01:19:41.891472+0000","last_clean":"2024-09-21T01:19:41.891472+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:19:41.891472+0000","last_undegraded":"2024-09-21T01:19:41.891472+0000","last_fullsized":"2024-09-21T01:19:41.891472+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.385979+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:04.385979+0000","last_peered":"2024-09-21T01:20:04.385979+0000","last_clean":"2024-09-21T01:20:04.385979+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:04.385979+0000","last_undegraded":"2024-09-21T01:20:04.385979+0000","last_fullsized":"2024-09-21T01:20:04.385979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.897797+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:19:41.897797+0000","last_peered":"2024-09-21T01:19:41.897797+0000","last_clean":"2024-09-21T01:19:41.897797+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:19:41.897797+0000","last_undegraded":"2024-09-21T01:19:41.897797+0000","last_fullsized":"2024-09-21T01:19:41.897797+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896004+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:19:41.896004+0000","last_peered":"2024-09-21T01:19:41.896004+0000","last_clean":"2024-09-21T01:19:41.896004+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:19:41.896004+0000","last_undegraded":"2024-09-21T01:19:41.896004+0000","last_fullsized":"2024-09-21T01:19:41.896004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.362702+0000","last_change":"2024-09-21T01:19:03.680497+0000","last_active":"2024-09-21T01:20:03.362702+0000","last_peered":"2024-09-21T01:20:03.362702+0000","last_clean":"2024-09-21T01:20:03.362702+0000","last_became_active":"2024-09-21T01:19:03.679642+0000","last_became_peered":"2024-09-21T01:19:03.679642+0000","last_unstale":"2024-09-21T01:20:03.362702+0000","last_undegraded":"2024-09-21T01:20:03.362702+0000","last_fullsized":"2024-09-21T01:20:03.362702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:25.218921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899464+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:19:41.899464+0000","last_peered":"2024-09-21T01:19:41.899464+0000","last_clean":"2024-09-21T01:19:41.899464+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:19:41.899464+0000","last_undegraded":"2024-09-21T01:19:41.899464+0000","last_fullsized":"2024-09-21T01:19:41.899464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899634+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:19:41.899634+0000","last_peered":"2024-09-21T01:19:41.899634+0000","last_clean":"2024-09-21T01:19:41.899634+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:19:41.899634+0000","last_undegraded":"2024-09-21T01:19:41.899634+0000","last_fullsized":"2024-09-21T01:19:41.899634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.387061+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:04.387061+0000","last_peered":"2024-09-21T01:20:04.387061+0000","last_clean":"2024-09-21T01:20:04.387061+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:04.387061+0000","last_undegraded":"2024-09-21T01:20:04.387061+0000","last_fullsized":"2024-09-21T01:20:04.387061+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.178050+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:19:42.178050+0000","last_peered":"2024-09-21T01:19:42.178050+0000","last_clean":"2024-09-21T01:19:42.178050+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:19:42.178050+0000","last_undegraded":"2024-09-21T01:19:42.178050+0000","last_fullsized":"2024-09-21T01:19:42.178050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890513+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:19:41.890513+0000","last_peered":"2024-09-21T01:19:41.890513+0000","last_clean":"2024-09-21T01:19:41.890513+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:19:41.890513+0000","last_undegraded":"2024-09-21T01:19:41.890513+0000","last_fullsized":"2024-09-21T01:19:41.890513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:23.428640+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:20:23.428640+0000","last_peered":"2024-09-21T01:20:23.428640+0000","last_clean":"2024-09-21T01:20:23.428640+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:20:23.428640+0000","last_undegraded":"2024-09-21T01:20:23.428640+0000","last_fullsized":"2024-09-21T01:20:23.428640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.104903+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:19:42.104903+0000","last_peered":"2024-09-21T01:19:42.104903+0000","last_clean":"2024-09-21T01:19:42.104903+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:19:42.104903+0000","last_undegraded":"2024-09-21T01:19:42.104903+0000","last_fullsized":"2024-09-21T01:19:42.104903+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891118+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:19:41.891118+0000","last_peered":"2024-09-21T01:19:41.891118+0000","last_clean":"2024-09-21T01:19:41.891118+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:19:41.891118+0000","last_undegraded":"2024-09-21T01:19:41.891118+0000","last_fullsized":"2024-09-21T01:19:41.891118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386210+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:04.386210+0000","last_peered":"2024-09-21T01:20:04.386210+0000","last_clean":"2024-09-21T01:20:04.386210+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:04.386210+0000","last_undegraded":"2024-09-21T01:20:04.386210+0000","last_fullsized":"2024-09-21T01:20:04.386210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895172+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:19:41.895172+0000","last_peered":"2024-09-21T01:19:41.895172+0000","last_clean":"2024-09-21T01:19:41.895172+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:19:41.895172+0000","last_undegraded":"2024-09-21T01:19:41.895172+0000","last_fullsized":"2024-09-21T01:19:41.895172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177425+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:19:42.177425+0000","last_peered":"2024-09-21T01:19:42.177425+0000","last_clean":"2024-09-21T01:19:42.177425+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:19:42.177425+0000","last_undegraded":"2024-09-21T01:19:42.177425+0000","last_fullsized":"2024-09-21T01:19:42.177425+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:05.397006+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:05.397006+0000","last_peered":"2024-09-21T01:20:05.397006+0000","last_clean":"2024-09-21T01:20:05.397006+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:05.397006+0000","last_undegraded":"2024-09-21T01:20:05.397006+0000","last_fullsized":"2024-09-21T01:20:05.397006+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899858+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:19:41.899858+0000","last_peered":"2024-09-21T01:19:41.899858+0000","last_clean":"2024-09-21T01:19:41.899858+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:19:41.899858+0000","last_undegraded":"2024-09-21T01:19:41.899858+0000","last_fullsized":"2024-09-21T01:19:41.899858+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890574+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:19:41.890574+0000","last_peered":"2024-09-21T01:19:41.890574+0000","last_clean":"2024-09-21T01:19:41.890574+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:19:41.890574+0000","last_undegraded":"2024-09-21T01:19:41.890574+0000","last_fullsized":"2024-09-21T01:19:41.890574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:24.430730+0000","last_change":"2024-09-21T01:19:03.684959+0000","last_active":"2024-09-21T01:20:24.430730+0000","last_peered":"2024-09-21T01:20:24.430730+0000","last_clean":"2024-09-21T01:20:24.430730+0000","last_became_active":"2024-09-21T01:19:03.684564+0000","last_became_peered":"2024-09-21T01:19:03.684564+0000","last_unstale":"2024-09-21T01:20:24.430730+0000","last_undegraded":"2024-09-21T01:20:24.430730+0000","last_fullsized":"2024-09-21T01:20:24.430730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:44:15.953025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60809,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":83,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898324+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:19:41.898324+0000","last_peered":"2024-09-21T01:19:41.898324+0000","last_clean":"2024-09-21T01:19:41.898324+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:19:41.898324+0000","last_undegraded":"2024-09-21T01:19:41.898324+0000","last_fullsized":"2024-09-21T01:19:41.898324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968951+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:19:41.968951+0000","last_peered":"2024-09-21T01:19:41.968951+0000","last_clean":"2024-09-21T01:19:41.968951+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:19:41.968951+0000","last_undegraded":"2024-09-21T01:19:41.968951+0000","last_fullsized":"2024-09-21T01:19:41.968951+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.381440+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:03.381440+0000","last_peered":"2024-09-21T01:20:03.381440+0000","last_clean":"2024-09-21T01:20:03.381440+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:03.381440+0000","last_undegraded":"2024-09-21T01:20:03.381440+0000","last_fullsized":"2024-09-21T01:20:03.381440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893899+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:19:41.893899+0000","last_peered":"2024-09-21T01:19:41.893899+0000","last_clean":"2024-09-21T01:19:41.893899+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:19:41.893899+0000","last_undegraded":"2024-09-21T01:19:41.893899+0000","last_fullsized":"2024-09-21T01:19:41.893899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893355+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:19:41.893355+0000","last_peered":"2024-09-21T01:19:41.893355+0000","last_clean":"2024-09-21T01:19:41.893355+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:19:41.893355+0000","last_undegraded":"2024-09-21T01:19:41.893355+0000","last_fullsized":"2024-09-21T01:19:41.893355+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.385149+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:04.385149+0000","last_peered":"2024-09-21T01:20:04.385149+0000","last_clean":"2024-09-21T01:20:04.385149+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:04.385149+0000","last_undegraded":"2024-09-21T01:20:04.385149+0000","last_fullsized":"2024-09-21T01:20:04.385149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896845+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:19:41.896845+0000","last_peered":"2024-09-21T01:19:41.896845+0000","last_clean":"2024-09-21T01:19:41.896845+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:19:41.896845+0000","last_undegraded":"2024-09-21T01:19:41.896845+0000","last_fullsized":"2024-09-21T01:19:41.896845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.969655+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:19:41.969655+0000","last_peered":"2024-09-21T01:19:41.969655+0000","last_clean":"2024-09-21T01:19:41.969655+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:19:41.969655+0000","last_undegraded":"2024-09-21T01:19:41.969655+0000","last_fullsized":"2024-09-21T01:19:41.969655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":85,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:24.430240+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:20:24.430240+0000","last_peered":"2024-09-21T01:20:24.430240+0000","last_clean":"2024-09-21T01:20:24.430240+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:20:24.430240+0000","last_undegraded":"2024-09-21T01:20:24.430240+0000","last_fullsized":"2024-09-21T01:20:24.430240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.102796+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:19:42.102796+0000","last_peered":"2024-09-21T01:19:42.102796+0000","last_clean":"2024-09-21T01:19:42.102796+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:19:42.102796+0000","last_undegraded":"2024-09-21T01:19:42.102796+0000","last_fullsized":"2024-09-21T01:19:42.102796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892829+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:19:41.892829+0000","last_peered":"2024-09-21T01:19:41.892829+0000","last_clean":"2024-09-21T01:19:41.892829+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:19:41.892829+0000","last_undegraded":"2024-09-21T01:19:41.892829+0000","last_fullsized":"2024-09-21T01:19:41.892829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.897128+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:19:41.897128+0000","last_peered":"2024-09-21T01:19:41.897128+0000","last_clean":"2024-09-21T01:19:41.897128+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:19:41.897128+0000","last_undegraded":"2024-09-21T01:19:41.897128+0000","last_fullsized":"2024-09-21T01:19:41.897128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:04.386310+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:04.386310+0000","last_peered":"2024-09-21T01:20:04.386310+0000","last_clean":"2024-09-21T01:20:04.386310+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:04.386310+0000","last_undegraded":"2024-09-21T01:20:04.386310+0000","last_fullsized":"2024-09-21T01:20:04.386310+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899821+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:19:41.899821+0000","last_peered":"2024-09-21T01:19:41.899821+0000","last_clean":"2024-09-21T01:19:41.899821+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:19:41.899821+0000","last_undegraded":"2024-09-21T01:19:41.899821+0000","last_fullsized":"2024-09-21T01:19:41.899821+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103631+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:19:42.103631+0000","last_peered":"2024-09-21T01:19:42.103631+0000","last_clean":"2024-09-21T01:19:42.103631+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:19:42.103631+0000","last_undegraded":"2024-09-21T01:19:42.103631+0000","last_fullsized":"2024-09-21T01:19:42.103631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:02.372616+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:02.372616+0000","last_peered":"2024-09-21T01:20:02.372616+0000","last_clean":"2024-09-21T01:20:02.372616+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:02.372616+0000","last_undegraded":"2024-09-21T01:20:02.372616+0000","last_fullsized":"2024-09-21T01:20:02.372616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:03.380740+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:03.380740+0000","last_peered":"2024-09-21T01:20:03.380740+0000","last_clean":"2024-09-21T01:20:03.380740+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:03.380740+0000","last_undegraded":"2024-09-21T01:20:03.380740+0000","last_fullsized":"2024-09-21T01:20:03.380740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.899457+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:19:41.899457+0000","last_peered":"2024-09-21T01:19:41.899457+0000","last_clean":"2024-09-21T01:19:41.899457+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:19:41.899457+0000","last_undegraded":"2024-09-21T01:19:41.899457+0000","last_fullsized":"2024-09-21T01:19:41.899457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895505+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:19:41.895505+0000","last_peered":"2024-09-21T01:19:41.895505+0000","last_clean":"2024-09-21T01:19:41.895505+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:19:41.895505+0000","last_undegraded":"2024-09-21T01:19:41.895505+0000","last_fullsized":"2024-09-21T01:19:41.895505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:20:20.424998+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:20:20.424998+0000","last_peered":"2024-09-21T01:20:20.424998+0000","last_clean":"2024-09-21T01:20:20.424998+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:20:20.424998+0000","last_undegraded":"2024-09-21T01:20:20.424998+0000","last_fullsized":"2024-09-21T01:20:20.424998+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890353+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:19:41.890353+0000","last_peered":"2024-09-21T01:19:41.890353+0000","last_clean":"2024-09-21T01:19:41.890353+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:19:41.890353+0000","last_undegraded":"2024-09-21T01:19:41.890353+0000","last_fullsized":"2024-09-21T01:19:41.890353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895959+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:19:41.895959+0000","last_peered":"2024-09-21T01:19:41.895959+0000","last_clean":"2024-09-21T01:19:41.895959+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:19:41.895959+0000","last_undegraded":"2024-09-21T01:19:41.895959+0000","last_fullsized":"2024-09-21T01:19:41.895959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.894868+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:19:41.894868+0000","last_peered":"2024-09-21T01:19:41.894868+0000","last_clean":"2024-09-21T01:19:41.894868+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:19:41.894868+0000","last_undegraded":"2024-09-21T01:19:41.894868+0000","last_fullsized":"2024-09-21T01:19:41.894868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.902359+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:19:41.902359+0000","last_peered":"2024-09-21T01:19:41.902359+0000","last_clean":"2024-09-21T01:19:41.902359+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:19:41.902359+0000","last_undegraded":"2024-09-21T01:19:41.902359+0000","last_fullsized":"2024-09-21T01:19:41.902359+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895565+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:19:41.895565+0000","last_peered":"2024-09-21T01:19:41.895565+0000","last_clean":"2024-09-21T01:19:41.895565+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:19:41.895565+0000","last_undegraded":"2024-09-21T01:19:41.895565+0000","last_fullsized":"2024-09-21T01:19:41.895565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893289+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:19:41.893289+0000","last_peered":"2024-09-21T01:19:41.893289+0000","last_clean":"2024-09-21T01:19:41.893289+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:19:41.893289+0000","last_undegraded":"2024-09-21T01:19:41.893289+0000","last_fullsized":"2024-09-21T01:19:41.893289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177820+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:19:42.177820+0000","last_peered":"2024-09-21T01:19:42.177820+0000","last_clean":"2024-09-21T01:19:42.177820+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:19:42.177820+0000","last_undegraded":"2024-09-21T01:19:42.177820+0000","last_fullsized":"2024-09-21T01:19:42.177820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896146+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:19:41.896146+0000","last_peered":"2024-09-21T01:19:41.896146+0000","last_clean":"2024-09-21T01:19:41.896146+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:19:41.896146+0000","last_undegraded":"2024-09-21T01:19:41.896146+0000","last_fullsized":"2024-09-21T01:19:41.896146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896250+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:19:41.896250+0000","last_peered":"2024-09-21T01:19:41.896250+0000","last_clean":"2024-09-21T01:19:41.896250+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:19:41.896250+0000","last_undegraded":"2024-09-21T01:19:41.896250+0000","last_fullsized":"2024-09-21T01:19:41.896250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103688+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:19:42.103688+0000","last_peered":"2024-09-21T01:19:42.103688+0000","last_clean":"2024-09-21T01:19:42.103688+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:19:42.103688+0000","last_undegraded":"2024-09-21T01:19:42.103688+0000","last_fullsized":"2024-09-21T01:19:42.103688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898015+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:19:41.898015+0000","last_peered":"2024-09-21T01:19:41.898015+0000","last_clean":"2024-09-21T01:19:41.898015+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:19:41.898015+0000","last_undegraded":"2024-09-21T01:19:41.898015+0000","last_fullsized":"2024-09-21T01:19:41.898015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.103042+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:19:42.103042+0000","last_peered":"2024-09-21T01:19:42.103042+0000","last_clean":"2024-09-21T01:19:42.103042+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:19:42.103042+0000","last_undegraded":"2024-09-21T01:19:42.103042+0000","last_fullsized":"2024-09-21T01:19:42.103042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.969128+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:19:41.969128+0000","last_peered":"2024-09-21T01:19:41.969128+0000","last_clean":"2024-09-21T01:19:41.969128+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:19:41.969128+0000","last_undegraded":"2024-09-21T01:19:41.969128+0000","last_fullsized":"2024-09-21T01:19:41.969128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177796+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:19:42.177796+0000","last_peered":"2024-09-21T01:19:42.177796+0000","last_clean":"2024-09-21T01:19:42.177796+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:19:42.177796+0000","last_undegraded":"2024-09-21T01:19:42.177796+0000","last_fullsized":"2024-09-21T01:19:42.177796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895308+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:19:41.895308+0000","last_peered":"2024-09-21T01:19:41.895308+0000","last_clean":"2024-09-21T01:19:41.895308+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:19:41.895308+0000","last_undegraded":"2024-09-21T01:19:41.895308+0000","last_fullsized":"2024-09-21T01:19:41.895308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896367+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:19:41.896367+0000","last_peered":"2024-09-21T01:19:41.896367+0000","last_clean":"2024-09-21T01:19:41.896367+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:19:41.896367+0000","last_undegraded":"2024-09-21T01:19:41.896367+0000","last_fullsized":"2024-09-21T01:19:41.896367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.898594+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:19:41.898594+0000","last_peered":"2024-09-21T01:19:41.898594+0000","last_clean":"2024-09-21T01:19:41.898594+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:19:41.898594+0000","last_undegraded":"2024-09-21T01:19:41.898594+0000","last_fullsized":"2024-09-21T01:19:41.898594+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896653+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:19:41.896653+0000","last_peered":"2024-09-21T01:19:41.896653+0000","last_clean":"2024-09-21T01:19:41.896653+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:19:41.896653+0000","last_undegraded":"2024-09-21T01:19:41.896653+0000","last_fullsized":"2024-09-21T01:19:41.896653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.968521+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:19:41.968521+0000","last_peered":"2024-09-21T01:19:41.968521+0000","last_clean":"2024-09-21T01:19:41.968521+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:19:41.968521+0000","last_undegraded":"2024-09-21T01:19:41.968521+0000","last_fullsized":"2024-09-21T01:19:41.968521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892640+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:19:41.892640+0000","last_peered":"2024-09-21T01:19:41.892640+0000","last_clean":"2024-09-21T01:19:41.892640+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:19:41.892640+0000","last_undegraded":"2024-09-21T01:19:41.892640+0000","last_fullsized":"2024-09-21T01:19:41.892640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892851+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:19:41.892851+0000","last_peered":"2024-09-21T01:19:41.892851+0000","last_clean":"2024-09-21T01:19:41.892851+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:19:41.892851+0000","last_undegraded":"2024-09-21T01:19:41.892851+0000","last_fullsized":"2024-09-21T01:19:41.892851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.177726+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:19:42.177726+0000","last_peered":"2024-09-21T01:19:42.177726+0000","last_clean":"2024-09-21T01:19:42.177726+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:19:42.177726+0000","last_undegraded":"2024-09-21T01:19:42.177726+0000","last_fullsized":"2024-09-21T01:19:42.177726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893877+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:19:41.893877+0000","last_peered":"2024-09-21T01:19:41.893877+0000","last_clean":"2024-09-21T01:19:41.893877+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:19:41.893877+0000","last_undegraded":"2024-09-21T01:19:41.893877+0000","last_fullsized":"2024-09-21T01:19:41.893877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.891364+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:19:41.891364+0000","last_peered":"2024-09-21T01:19:41.891364+0000","last_clean":"2024-09-21T01:19:41.891364+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:19:41.891364+0000","last_undegraded":"2024-09-21T01:19:41.891364+0000","last_fullsized":"2024-09-21T01:19:41.891364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896628+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:19:41.896628+0000","last_peered":"2024-09-21T01:19:41.896628+0000","last_clean":"2024-09-21T01:19:41.896628+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:19:41.896628+0000","last_undegraded":"2024-09-21T01:19:41.896628+0000","last_fullsized":"2024-09-21T01:19:41.896628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.969655+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:19:41.969655+0000","last_peered":"2024-09-21T01:19:41.969655+0000","last_clean":"2024-09-21T01:19:41.969655+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:19:41.969655+0000","last_undegraded":"2024-09-21T01:19:41.969655+0000","last_fullsized":"2024-09-21T01:19:41.969655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:42.102966+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:19:42.102966+0000","last_peered":"2024-09-21T01:19:42.102966+0000","last_clean":"2024-09-21T01:19:42.102966+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:19:42.102966+0000","last_undegraded":"2024-09-21T01:19:42.102966+0000","last_fullsized":"2024-09-21T01:19:42.102966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890676+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:19:41.890676+0000","last_peered":"2024-09-21T01:19:41.890676+0000","last_clean":"2024-09-21T01:19:41.890676+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:19:41.890676+0000","last_undegraded":"2024-09-21T01:19:41.890676+0000","last_fullsized":"2024-09-21T01:19:41.890676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.895930+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:19:41.895930+0000","last_peered":"2024-09-21T01:19:41.895930+0000","last_clean":"2024-09-21T01:19:41.895930+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:19:41.895930+0000","last_undegraded":"2024-09-21T01:19:41.895930+0000","last_fullsized":"2024-09-21T01:19:41.895930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.890219+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:19:41.890219+0000","last_peered":"2024-09-21T01:19:41.890219+0000","last_clean":"2024-09-21T01:19:41.890219+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:19:41.890219+0000","last_undegraded":"2024-09-21T01:19:41.890219+0000","last_fullsized":"2024-09-21T01:19:41.890219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893753+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:19:41.893753+0000","last_peered":"2024-09-21T01:19:41.893753+0000","last_clean":"2024-09-21T01:19:41.893753+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:19:41.893753+0000","last_undegraded":"2024-09-21T01:19:41.893753+0000","last_fullsized":"2024-09-21T01:19:41.893753+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.893340+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:19:41.893340+0000","last_peered":"2024-09-21T01:19:41.893340+0000","last_clean":"2024-09-21T01:19:41.893340+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:19:41.893340+0000","last_undegraded":"2024-09-21T01:19:41.893340+0000","last_fullsized":"2024-09-21T01:19:41.893340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.892982+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:19:41.892982+0000","last_peered":"2024-09-21T01:19:41.892982+0000","last_clean":"2024-09-21T01:19:41.892982+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:19:41.892982+0000","last_undegraded":"2024-09-21T01:19:41.892982+0000","last_fullsized":"2024-09-21T01:19:41.892982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-21T01:19:41.896526+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:19:41.896526+0000","last_peered":"2024-09-21T01:19:41.896526+0000","last_clean":"2024-09-21T01:19:41.896526+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:19:41.896526+0000","last_undegraded":"2024-09-21T01:19:41.896526+0000","last_fullsized":"2024-09-21T01:19:41.896526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68013,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":232,"num_read_kb":193,"num_write":245,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":195509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":516031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874275,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28696,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716456,"statfs":{"total":95995035648,"available":95965650944,"internally_reserved":0,"allocated":1712128,"data_stored":1168143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037800,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":515961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":516339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1113227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":511465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790016,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1108418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":515930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1028096,"data_stored":511173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":569650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542229,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":569039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705756,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":512921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:20:28.714 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T01:20:28.714 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T01:20:28.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T01:20:28.800 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:28 smithi027 ceph-mon[30375]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:20:28.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:28 smithi098 ceph-mon[34372]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:20:29.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:28 smithi089 ceph-mon[34486]: from='client.24992 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:20:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 -- 172.21.15.27:0/1984160004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c077380 msgr2=0x7f304c0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 --2- 172.21.15.27:0/1984160004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c077380 0x7f304c0757b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3040009900 tx=0x7f304002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 -- 172.21.15.27:0/1984160004 shutdown_connections 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 --2- 172.21.15.27:0/1984160004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c114440 0x7f304c116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 --2- 172.21.15.27:0/1984160004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c075dd0 0x7f304c0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 --2- 172.21.15.27:0/1984160004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c077380 0x7f304c0757b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 -- 172.21.15.27:0/1984160004 >> 172.21.15.27:0/1984160004 conn(0x7f304c0ff5a0 msgr2=0x7f304c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:29.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 -- 172.21.15.27:0/1984160004 shutdown_connections 2024-09-21T01:20:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.817+0000 7f3053a00640 1 -- 172.21.15.27:0/1984160004 wait complete. 2024-09-21T01:20:29.819 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:29 smithi027 ceph-mon[30375]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-21T01:20:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.818+0000 7f3053a00640 1 Processor -- start 2024-09-21T01:20:29.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 -- start start 2024-09-21T01:20:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c075dd0 0x7f304c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:29.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 0x7f304c1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c114440 0x7f304c1a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f304c11a0c0 con 0x7f304c075dd0 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f304c119f40 con 0x7f304c114440 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f3053a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f304c11a240 con 0x7f304c077380 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f304be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 0x7f304c1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f304be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 0x7f304c1a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:52366/0 (socket says 172.21.15.27:52366) 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.819+0000 7f304be00640 1 -- 172.21.15.27:0/1663953198 learned_addr learned my addr 172.21.15.27:0/1663953198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f3051000640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c075dd0 0x7f304c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f3051a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c114440 0x7f304c1a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f304be00640 1 -- 172.21.15.27:0/1663953198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c114440 msgr2=0x7f304c1a6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f304be00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c114440 0x7f304c1a6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f304be00640 1 -- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c075dd0 msgr2=0x7f304c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f304be00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c075dd0 0x7f304c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:29.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f304be00640 1 -- 172.21.15.27:0/1663953198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c1a6a40 con 0x7f304c077380 2024-09-21T01:20:29.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.820+0000 7f304be00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 0x7f304c1a1070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f303800e9c0 tx=0x7f303800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.821+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303800ccc0 con 0x7f304c077380 2024-09-21T01:20:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.821+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30380054d0 con 0x7f304c077380 2024-09-21T01:20:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.821+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038010430 con 0x7f304c077380 2024-09-21T01:20:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.821+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c1a6d30 con 0x7f304c077380 2024-09-21T01:20:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.821+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1ad490 con 0x7f304c077380 2024-09-21T01:20:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.822+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c0761e0 con 0x7f304c077380 2024-09-21T01:20:29.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.825+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3038004f20 con 0x7f304c077380 2024-09-21T01:20:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.827+0000 7f3049600640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30340b6670 0x7f30340b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.827+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f30380e81b0 con 0x7f304c077380 2024-09-21T01:20:29.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.827+0000 7f3051000640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30340b6670 0x7f30340b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.827+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30380ae460 con 0x7f304c077380 2024-09-21T01:20:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:29.828+0000 7f3051000640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30340b6670 0x7f30340b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3040005ab0 tx=0x7f3040005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:29.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:29 smithi098 ceph-mon[34372]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-21T01:20:30.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:29 smithi089 ceph-mon[34486]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-21T01:20:30.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.101+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f304c02e640 con 0x7f304c077380 2024-09-21T01:20:30.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.111+0000 7f3049600640 1 -- 172.21.15.27:0/1663953198 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f30380b2840 con 0x7f304c077380 2024-09-21T01:20:30.111 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-09-21T01:20:30.111 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBeH+5mEJ4vBhAATTs21CfxB1MEkCANPGoTeg== 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30340b6670 msgr2=0x7f30340b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30340b6670 0x7f30340b8b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3040005ab0 tx=0x7f3040005a20 comp rx=0 tx=0).stop 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 msgr2=0x7f304c1a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 0x7f304c1a1070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f303800e9c0 tx=0x7f303800edc0 comp rx=0 tx=0).stop 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 shutdown_connections 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f30340b6670 0x7f30340b8b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:30.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f304c114440 0x7f304c1a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:30.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f304c077380 0x7f304c1a1070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:30.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 --2- 172.21.15.27:0/1663953198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f304c075dd0 0x7f304c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:30.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 >> 172.21.15.27:0/1663953198 conn(0x7f304c0ff5a0 msgr2=0x7f304c110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:30.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 shutdown_connections 2024-09-21T01:20:30.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:30.113+0000 7f3053a00640 1 -- 172.21.15.27:0/1663953198 wait complete. 2024-09-21T01:20:30.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:30 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1663953198' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:20:30.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:30 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:20:30.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:30 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T01:20:30.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:30 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1663953198' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:20:30.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:30 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:20:30.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:30 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T01:20:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:30 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1663953198' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:20:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:30 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T01:20:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:30 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T01:20:31.078 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:20:31.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T01:20:31.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T01:20:31.162 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T01:20:31.588 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 -- 172.21.15.27:0/155361846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 msgr2=0x7f8e8010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 --2- 172.21.15.27:0/155361846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 0x7f8e8010bad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009900 tx=0x7f8e6c02a410 comp rx=0 tx=0).stop 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 -- 172.21.15.27:0/155361846 shutdown_connections 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 --2- 172.21.15.27:0/155361846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e8010c010 0x7f8e80112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 --2- 172.21.15.27:0/155361846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 0x7f8e8010bad0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 --2- 172.21.15.27:0/155361846 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e80106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 -- 172.21.15.27:0/155361846 >> 172.21.15.27:0/155361846 conn(0x7f8e80101810 msgr2=0x7f8e80103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 -- 172.21.15.27:0/155361846 shutdown_connections 2024-09-21T01:20:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.217+0000 7f8e88400640 1 -- 172.21.15.27:0/155361846 wait complete. 2024-09-21T01:20:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.218+0000 7f8e88400640 1 Processor -- start 2024-09-21T01:20:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.218+0000 7f8e88400640 1 -- start start 2024-09-21T01:20:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e88400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e8019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e88400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 0x7f8e8019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e88400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e8010c010 0x7f8e801a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e88400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80115d20 con 0x7f8e8010b6c0 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e88400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80115ba0 con 0x7f8e80105d80 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e88400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80115ea0 con 0x7f8e8010c010 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e8019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e8019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58898/0 (socket says 172.21.15.27:58898) 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e85000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 0x7f8e8019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e85a00640 1 -- 172.21.15.27:0/2416285520 learned_addr learned my addr 172.21.15.27:0/2416285520 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.219+0000 7f8e86400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e8010c010 0x7f8e801a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e85a00640 1 -- 172.21.15.27:0/2416285520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e8010c010 msgr2=0x7f8e801a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e85a00640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e8010c010 0x7f8e801a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e85a00640 1 -- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 msgr2=0x7f8e8019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e85a00640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 0x7f8e8019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e85a00640 1 -- 172.21.15.27:0/2416285520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e801a2620 con 0x7f8e80105d80 2024-09-21T01:20:32.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e85a00640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e8019c6b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e9f0 tx=0x7f8e7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400cd80 con 0x7f8e80105d80 2024-09-21T01:20:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e740108e0 con 0x7f8e80105d80 2024-09-21T01:20:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.220+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e801a2910 con 0x7f8e80105d80 2024-09-21T01:20:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.221+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74018980 con 0x7f8e80105d80 2024-09-21T01:20:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.221+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e801a9070 con 0x7f8e80105d80 2024-09-21T01:20:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.222+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e80106190 con 0x7f8e80105d80 2024-09-21T01:20:32.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.224+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8e74010430 con 0x7f8e80105d80 2024-09-21T01:20:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.225+0000 7f8e7e000640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8e680b65f0 0x7f8e680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.226+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f8e740e80f0 con 0x7f8e80105d80 2024-09-21T01:20:32.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.226+0000 7f8e85000640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8e680b65f0 0x7f8e680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.226+0000 7f8e85000640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8e680b65f0 0x7f8e680b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02aa00 tx=0x7f8e6c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:32.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.227+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e740b2780 con 0x7f8e80105d80 2024-09-21T01:20:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:32 smithi098 ceph-mon[34372]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-21T01:20:32.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.424+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f8e8019d7e0 con 0x7f8e80105d80 2024-09-21T01:20:32.425 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:32 smithi027 ceph-mon[30375]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-21T01:20:32.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.425+0000 7f8e7e000640 1 -- 172.21.15.27:0/2416285520 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f8e740b76c0 con 0x7f8e80105d80 2024-09-21T01:20:32.425 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8e680b65f0 msgr2=0x7f8e680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8e680b65f0 0x7f8e680b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02aa00 tx=0x7f8e6c005a20 comp rx=0 tx=0).stop 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 msgr2=0x7f8e8019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e8019c6b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e9f0 tx=0x7f8e7400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 shutdown_connections 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8e680b65f0 0x7f8e680b8b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e8010c010 0x7f8e801a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e8010b6c0 0x7f8e8019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 --2- 172.21.15.27:0/2416285520 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e80105d80 0x7f8e8019c6b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 >> 172.21.15.27:0/2416285520 conn(0x7f8e80101810 msgr2=0x7f8e8010ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 shutdown_connections 2024-09-21T01:20:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:32.428+0000 7f8e88400640 1 -- 172.21.15.27:0/2416285520 wait complete. 2024-09-21T01:20:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:32 smithi089 ceph-mon[34486]: pgmap v341: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-09-21T01:20:33.345 INFO:teuthology.orchestra.run.smithi027.stdout:[{"version":55,"timestamp":"2024-09-21T01:18:54.339654+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T01:20:33.346 INFO:tasks.ceph_manager:config epoch is 55 2024-09-21T01:20:33.346 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T01:20:33.346 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T01:20:33.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T01:20:33.397 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:33 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2416285520' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T01:20:33.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:33 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2416285520' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T01:20:33.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:33 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2416285520' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T01:20:33.737 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:34 smithi027 ceph-mon[30375]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T01:20:34.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:34 smithi098 ceph-mon[34372]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 -- 172.21.15.27:0/3313746197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050075dd0 msgr2=0x7f10500761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 --2- 172.21.15.27:0/3313746197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050075dd0 0x7f10500761e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f103c009900 tx=0x7f103c02a410 comp rx=0 tx=0).stop 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 -- 172.21.15.27:0/3313746197 shutdown_connections 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 --2- 172.21.15.27:0/3313746197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1050114280 0x7f1050116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 --2- 172.21.15.27:0/3313746197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050075dd0 0x7f10500761e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 --2- 172.21.15.27:0/3313746197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10500757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 -- 172.21.15.27:0/3313746197 >> 172.21.15.27:0/3313746197 conn(0x7f10500ff440 msgr2=0x7f1050101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:34.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 -- 172.21.15.27:0/3313746197 shutdown_connections 2024-09-21T01:20:34.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.445+0000 7f104f200640 1 -- 172.21.15.27:0/3313746197 wait complete. 2024-09-21T01:20:34.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.446+0000 7f104f200640 1 Processor -- start 2024-09-21T01:20:34.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 -- start start 2024-09-21T01:20:34.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1050075dd0 0x7f10501a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:34.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10501a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:34.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050114280 0x7f10501a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1050119ec0 con 0x7f1050114280 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1050119d40 con 0x7f1050077380 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f105011a040 con 0x7f1050075dd0 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10501a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10501a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58928/0 (socket says 172.21.15.27:58928) 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104d400640 1 -- 172.21.15.27:0/645743667 learned_addr learned my addr 172.21.15.27:0/645743667 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.447+0000 7f104de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1050075dd0 0x7f10501a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104e800640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050114280 0x7f10501a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104d400640 1 -- 172.21.15.27:0/645743667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1050075dd0 msgr2=0x7f10501a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104d400640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1050075dd0 0x7f10501a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104d400640 1 -- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050114280 msgr2=0x7f10501a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104d400640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050114280 0x7f10501a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104d400640 1 -- 172.21.15.27:0/645743667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10501a68c0 con 0x7f1050077380 2024-09-21T01:20:34.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.448+0000 7f104d400640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10501a0ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f103c032fd0 tx=0x7f103c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:34.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.449+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c03f030 con 0x7f1050077380 2024-09-21T01:20:34.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.449+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10501a6b50 con 0x7f1050077380 2024-09-21T01:20:34.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.449+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103c004490 con 0x7f1050077380 2024-09-21T01:20:34.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.449+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c035070 con 0x7f1050077380 2024-09-21T01:20:34.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.449+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501ad310 con 0x7f1050077380 2024-09-21T01:20:34.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.450+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1050076cc0 con 0x7f1050077380 2024-09-21T01:20:34.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.452+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f103c042030 con 0x7f1050077380 2024-09-21T01:20:34.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.454+0000 7f1046a00640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10340b65f0 0x7f10340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:34.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.454+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f103c107ff0 con 0x7f1050077380 2024-09-21T01:20:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.454+0000 7f104de00640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10340b65f0 0x7f10340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:34.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.455+0000 7f104de00640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10340b65f0 0x7f10340b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10400095f0 tx=0x7f1040005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:34.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.456+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f103c0d26e0 con 0x7f1050077380 2024-09-21T01:20:34.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:34 smithi089 ceph-mon[34486]: pgmap v342: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T01:20:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.705+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1050112fb0 con 0x7f1050077380 2024-09-21T01:20:34.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.711+0000 7f1046a00640 1 -- 172.21.15.27:0/645743667 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v22) ==== 74+0+296447 (secure 0 0 0) 0x7f103c0d7620 con 0x7f1050077380 2024-09-21T01:20:34.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:34.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.718+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10340b65f0 msgr2=0x7f10340b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:34.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.718+0000 7f104f200640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10340b65f0 0x7f10340b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f10400095f0 tx=0x7f1040005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:34.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.718+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 msgr2=0x7f10501a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:34.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.718+0000 7f104f200640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10501a0ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f103c032fd0 tx=0x7f103c031a20 comp rx=0 tx=0).stop 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 shutdown_connections 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10340b65f0 0x7f10340b8b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1050114280 0x7f10501a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1050077380 0x7f10501a0ef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 --2- 172.21.15.27:0/645743667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1050075dd0 0x7f10501a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 >> 172.21.15.27:0/645743667 conn(0x7f10500ff440 msgr2=0x7f1050110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 shutdown_connections 2024-09-21T01:20:34.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:34.719+0000 7f104f200640 1 -- 172.21.15.27:0/645743667 wait complete. 2024-09-21T01:20:35.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:35 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/645743667' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T01:20:35.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:35 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/645743667' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T01:20:35.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:35 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/645743667' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T01:20:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"flags":0,"active_gid":24110,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":39380413},{"type":"v1","addr":"172.21.15.27:6801","nonce":39380413}]},"active_addr":"172.21.15.27:6801/39380413","active_change":"2024-09-21T01:11:40.307023+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24190,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24200,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.27:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1753540833}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1131893290}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3904109230}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":3404006079}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":545846301}]}]} 2024-09-21T01:20:35.640 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T01:20:35.640 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T01:20:35.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:20:36.059 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:36.383 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:36 smithi027 ceph-mon[30375]: pgmap v343: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T01:20:36.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:36 smithi098 ceph-mon[34372]: pgmap v343: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T01:20:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:36 smithi089 ceph-mon[34486]: pgmap v343: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 -- 172.21.15.27:0/1476375731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c105d80 msgr2=0x7ffa6c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 --2- 172.21.15.27:0/1476375731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c105d80 0x7ffa6c106190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009860 tx=0x7ffa5c02a340 comp rx=0 tx=0).stop 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 -- 172.21.15.27:0/1476375731 shutdown_connections 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 --2- 172.21.15.27:0/1476375731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa6c10c010 0x7ffa6c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 --2- 172.21.15.27:0/1476375731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 --2- 172.21.15.27:0/1476375731 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c105d80 0x7ffa6c106190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.729+0000 7ffa72200640 1 -- 172.21.15.27:0/1476375731 >> 172.21.15.27:0/1476375731 conn(0x7ffa6c101810 msgr2=0x7ffa6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:36.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.730+0000 7ffa72200640 1 -- 172.21.15.27:0/1476375731 shutdown_connections 2024-09-21T01:20:36.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.730+0000 7ffa72200640 1 -- 172.21.15.27:0/1476375731 wait complete. 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 Processor -- start 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 -- start start 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 0x7ffa6c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa6c10c010 0x7ffa6c1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c115cd0 con 0x7ffa6c105d80 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c115b50 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.731+0000 7ffa72200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c115e50 con 0x7ffa6c10c010 2024-09-21T01:20:36.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 0x7ffa6c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43122/0 (socket says 172.21.15.27:43122) 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 0x7ffa6c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55560/0 (socket says 172.21.15.27:55560) 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 -- 172.21.15.27:0/2704904926 learned_addr learned my addr 172.21.15.27:0/2704904926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa6be00640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa6c10c010 0x7ffa6c1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 -- 172.21.15.27:0/2704904926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa6c10c010 msgr2=0x7ffa6c1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa6c10c010 0x7ffa6c1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 -- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 msgr2=0x7ffa6c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 0x7ffa6c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 -- 172.21.15.27:0/2704904926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c1a2610 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa6b400640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 0x7ffa6c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:36.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.732+0000 7ffa62a00640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c19cc40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400e9c0 tx=0x7ffa5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:36.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.733+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400ccc0 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.733+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa540108e0 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.733+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540187e0 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.733+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c1a2900 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.733+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c1a9060 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.734+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c106190 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.737+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ffa54010430 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.739+0000 7ffa68c00640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffa500b65f0 0x7ffa500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:36.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.739+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7ffa540e7ff0 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.739+0000 7ffa6b400640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffa500b65f0 0x7ffa500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:36.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.740+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffa540b26e0 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.740+0000 7ffa6b400640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffa500b65f0 0x7ffa500b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c02a930 tx=0x7ffa5c0058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:36.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.943+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa6c19d7d0 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.944+0000 7ffa68c00640 1 -- 172.21.15.27:0/2704904926 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24907 (secure 0 0 0) 0x7ffa540b7620 con 0x7ffa6c10b6c0 2024-09-21T01:20:36.946 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:36.946 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:41.886071+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:20:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffa500b65f0 msgr2=0x7ffa500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffa500b65f0 0x7ffa500b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c02a930 tx=0x7ffa5c0058d0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 msgr2=0x7ffa6c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c19cc40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400e9c0 tx=0x7ffa5400ee80 comp rx=0 tx=0).stop 2024-09-21T01:20:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 shutdown_connections 2024-09-21T01:20:36.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffa500b65f0 0x7ffa500b8b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa6c10c010 0x7ffa6c1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa6c10b6c0 0x7ffa6c19cc40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 --2- 172.21.15.27:0/2704904926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa6c105d80 0x7ffa6c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.947+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 >> 172.21.15.27:0/2704904926 conn(0x7ffa6c101810 msgr2=0x7ffa6c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.948+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 shutdown_connections 2024-09-21T01:20:36.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:36.948+0000 7ffa72200640 1 -- 172.21.15.27:0/2704904926 wait complete. 2024-09-21T01:20:37.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:37 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2704904926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:37.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:37 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2704904926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:37.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:37 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2704904926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:37.832 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T01:20:37.832 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:20:38.218 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:38 smithi098 ceph-mon[34372]: pgmap v344: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T01:20:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:38 smithi027 ceph-mon[30375]: pgmap v344: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T01:20:38.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:38 smithi089 ceph-mon[34486]: pgmap v344: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T01:20:38.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 -- 172.21.15.27:0/678993299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e801039b0 msgr2=0x7f1e80103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 --2- 172.21.15.27:0/678993299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e801039b0 0x7f1e80103dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009900 tx=0x7f1e7002a410 comp rx=0 tx=0).stop 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 -- 172.21.15.27:0/678993299 shutdown_connections 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 --2- 172.21.15.27:0/678993299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e80109c40 0x7f1e80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 --2- 172.21.15.27:0/678993299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e801092f0 0x7f1e80109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 --2- 172.21.15.27:0/678993299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e801039b0 0x7f1e80103dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 -- 172.21.15.27:0/678993299 >> 172.21.15.27:0/678993299 conn(0x7f1e800ff440 msgr2=0x7f1e80101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.853+0000 7f1e86000640 1 -- 172.21.15.27:0/678993299 shutdown_connections 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.854+0000 7f1e86000640 1 -- 172.21.15.27:0/678993299 wait complete. 2024-09-21T01:20:38.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.855+0000 7f1e86000640 1 Processor -- start 2024-09-21T01:20:38.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.855+0000 7f1e86000640 1 -- start start 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e86000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 0x7f1e8019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e86000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e801092f0 0x7f1e8019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e86000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e80109c40 0x7f1e8019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e86000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80113900 con 0x7f1e80109c40 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e86000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80113780 con 0x7f1e801092f0 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e86000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80113a80 con 0x7f1e801039b0 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e85600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e80109c40 0x7f1e8019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e84c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 0x7f1e8019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e84c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 0x7f1e8019a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51632/0 (socket says 172.21.15.27:51632) 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e84c00640 1 -- 172.21.15.27:0/1722137214 learned_addr learned my addr 172.21.15.27:0/1722137214 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:38.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.856+0000 7f1e77e00640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e801092f0 0x7f1e8019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e84c00640 1 -- 172.21.15.27:0/1722137214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e801092f0 msgr2=0x7f1e8019a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e84c00640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e801092f0 0x7f1e8019a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e84c00640 1 -- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e80109c40 msgr2=0x7f1e8019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e84c00640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e80109c40 0x7f1e8019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:38.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e84c00640 1 -- 172.21.15.27:0/1722137214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e801a0280 con 0x7f1e801039b0 2024-09-21T01:20:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e84c00640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 0x7f1e8019a310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e70005940 tx=0x7f1e700040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003f030 con 0x7f1e801039b0 2024-09-21T01:20:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801a0510 con 0x7f1e801039b0 2024-09-21T01:20:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e700044c0 con 0x7f1e801039b0 2024-09-21T01:20:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70035070 con 0x7f1e801039b0 2024-09-21T01:20:38.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.857+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e801a6cd0 con 0x7f1e801039b0 2024-09-21T01:20:38.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.858+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e80103dc0 con 0x7f1e801039b0 2024-09-21T01:20:38.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.861+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f1e70042030 con 0x7f1e801039b0 2024-09-21T01:20:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.862+0000 7f1e7e000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1e640b65f0 0x7f1e640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.863+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f1e70107f50 con 0x7f1e801039b0 2024-09-21T01:20:38.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.863+0000 7f1e77e00640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1e640b65f0 0x7f1e640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:38.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.864+0000 7f1e77e00640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1e640b65f0 0x7f1e640b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e80108e90 tx=0x7f1e68005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:38.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:38.864+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e700d25e0 con 0x7f1e801039b0 2024-09-21T01:20:39.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.068+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e8019b440 con 0x7f1e801039b0 2024-09-21T01:20:39.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.070+0000 7f1e7e000640 1 -- 172.21.15.27:0/1722137214 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24907 (secure 0 0 0) 0x7f1e700d7520 con 0x7f1e801039b0 2024-09-21T01:20:39.071 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:20:39.071 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":88,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:19:41.886071+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:20:39.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1e640b65f0 msgr2=0x7f1e640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1e640b65f0 0x7f1e640b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e80108e90 tx=0x7f1e68005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 msgr2=0x7f1e8019a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 0x7f1e8019a310 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e70005940 tx=0x7f1e700040c0 comp rx=0 tx=0).stop 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 shutdown_connections 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f1e640b65f0 0x7f1e640b8b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e80109c40 0x7f1e8019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e801092f0 0x7f1e8019a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 --2- 172.21.15.27:0/1722137214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1e801039b0 0x7f1e8019a310 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.072+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 >> 172.21.15.27:0/1722137214 conn(0x7f1e800ff440 msgr2=0x7f1e80101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.073+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 shutdown_connections 2024-09-21T01:20:39.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:39.073+0000 7f1e86000640 1 -- 172.21.15.27:0/1722137214 wait complete. 2024-09-21T01:20:39.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:39 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1722137214' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:39.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:39 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1722137214' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:39.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:39 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1722137214' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:20:40.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T01:20:40.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T01:20:40.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T01:20:40.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T01:20:40.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T01:20:40.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T01:20:40.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T01:20:40.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T01:20:40.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T01:20:40.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T01:20:40.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T01:20:40.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T01:20:40.186 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:40 smithi027 ceph-mon[30375]: pgmap v345: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:20:40.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:40 smithi089 ceph-mon[34486]: pgmap v345: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:20:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:40 smithi098 ceph-mon[34372]: pgmap v345: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T01:20:41.012 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:41.176 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T01:20:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T01:20:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T01:20:41.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:41 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T01:20:41.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T01:20:41.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T01:20:41.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T01:20:41.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:41 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T01:20:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-21T01:20:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T01:20:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T01:20:41.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:41 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-09-21T01:20:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:41.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:41.716 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:41.717 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:42.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:42 smithi027 ceph-mon[30375]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T01:20:42.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T01:20:42.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T01:20:42.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T01:20:42.266 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:42 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T01:20:42.267 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:42 smithi027 ceph-mon[30375]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.011+0000 7efd77e00640 1 -- 172.21.15.27:0/3443210586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 msgr2=0x7efd78110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.011+0000 7efd77e00640 1 --2- 172.21.15.27:0/3443210586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 0x7efd78110470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd70009ca0 tx=0x7efd7002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 -- 172.21.15.27:0/3443210586 shutdown_connections 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 --2- 172.21.15.27:0/3443210586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 0x7efd78110470 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 --2- 172.21.15.27:0/3443210586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781092f0 0x7efd78109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 --2- 172.21.15.27:0/3443210586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd781039b0 0x7efd78103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 -- 172.21.15.27:0/3443210586 >> 172.21.15.27:0/3443210586 conn(0x7efd780ff440 msgr2=0x7efd78101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 -- 172.21.15.27:0/3443210586 shutdown_connections 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.012+0000 7efd77e00640 1 -- 172.21.15.27:0/3443210586 wait complete. 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.013+0000 7efd77e00640 1 Processor -- start 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.013+0000 7efd77e00640 1 -- start start 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.013+0000 7efd77e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781039b0 0x7efd7819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd77e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd781092f0 0x7efd7819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 0x7efd781a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd77e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd78113900 con 0x7efd78109c40 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd77e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd78113780 con 0x7efd781092f0 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd77e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd78113a80 con 0x7efd781039b0 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781039b0 0x7efd7819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781039b0 0x7efd7819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51652/0 (socket says 172.21.15.27:51652) 2024-09-21T01:20:42.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 -- 172.21.15.27:0/2046687642 learned_addr learned my addr 172.21.15.27:0/2046687642 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd77400640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 0x7efd781a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76000640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd781092f0 0x7efd7819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 -- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd781092f0 msgr2=0x7efd7819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd781092f0 0x7efd7819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 -- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 msgr2=0x7efd781a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd78109c40 0x7efd781a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.014+0000 7efd76a00640 1 -- 172.21.15.27:0/2046687642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd781a2420 con 0x7efd781039b0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.015+0000 7efd76a00640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781039b0 0x7efd7819c4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd6800ea60 tx=0x7efd6800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.015+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6800cde0 con 0x7efd781039b0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.015+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd781a2710 con 0x7efd781039b0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.015+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd680035f0 con 0x7efd781039b0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.015+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68010690 con 0x7efd781039b0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.015+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd781a8e70 con 0x7efd781039b0 2024-09-21T01:20:42.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.016+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd781101a0 con 0x7efd781039b0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.019+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7efd680037a0 con 0x7efd781039b0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd6f400640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd5c0b6670 0x7efd5c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7efd680e8290 con 0x7efd781039b0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd76000640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd5c0b6670 0x7efd5c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd6f400640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] conn(0x7efd5c0c1b80 0x7efd5c0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 --> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd6800be50 con 0x7efd5c0c1b80 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7efd680e8640 con 0x7efd781039b0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.021+0000 7efd77400640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] conn(0x7efd5c0c1b80 0x7efd5c0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.022+0000 7efd76000640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd5c0b6670 0x7efd5c0b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd78105cf0 tx=0x7efd64005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.022+0000 7efd77400640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] conn(0x7efd5c0c1b80 0x7efd5c0c4070 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.023+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== osd.4 v2:172.21.15.89:6808/2496831965 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd6800be50 con 0x7efd5c0c1b80 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.038+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 --> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd78104c30 con 0x7efd5c0c1b80 2024-09-21T01:20:42.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.038+0000 7efd6f400640 1 -- 172.21.15.27:0/2046687642 <== osd.4 v2:172.21.15.89:6808/2496831965 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd78104c30 con 0x7efd5c0c1b80 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] conn(0x7efd5c0c1b80 msgr2=0x7efd5c0c4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] conn(0x7efd5c0c1b80 0x7efd5c0c4070 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd5c0b6670 msgr2=0x7efd5c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd5c0b6670 0x7efd5c0b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd78105cf0 tx=0x7efd64005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781039b0 msgr2=0x7efd7819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 --2- 172.21.15.27:0/2046687642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd781039b0 0x7efd7819c4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd6800ea60 tx=0x7efd6800ef20 comp rx=0 tx=0).stop 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77400640 1 -- 172.21.15.27:0/2046687642 reap_dead start 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 shutdown_connections 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 >> 172.21.15.27:0/2046687642 conn(0x7efd780ff440 msgr2=0x7efd7810e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:42.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 shutdown_connections 2024-09-21T01:20:42.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.039+0000 7efd77e00640 1 -- 172.21.15.27:0/2046687642 wait complete. 2024-09-21T01:20:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:42 smithi089 ceph-mon[34486]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T01:20:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T01:20:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T01:20:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T01:20:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:42 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T01:20:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:42 smithi089 ceph-mon[34486]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T01:20:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:42 smithi098 ceph-mon[34372]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-21T01:20:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-21T01:20:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T01:20:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T01:20:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:42 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-21T01:20:42.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:42 smithi098 ceph-mon[34372]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T01:20:42.775 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 -- 172.21.15.27:0/545896162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c10c010 msgr2=0x7f0b5c112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 --2- 172.21.15.27:0/545896162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c10c010 0x7f0b5c112840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009be0 tx=0x7f0b5402a6c0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 -- 172.21.15.27:0/545896162 shutdown_connections 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 --2- 172.21.15.27:0/545896162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c10c010 0x7f0b5c112840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 --2- 172.21.15.27:0/545896162 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b5c10b6c0 0x7f0b5c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 --2- 172.21.15.27:0/545896162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b5c105d80 0x7f0b5c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 -- 172.21.15.27:0/545896162 >> 172.21.15.27:0/545896162 conn(0x7f0b5c101810 msgr2=0x7f0b5c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:42.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 -- 172.21.15.27:0/545896162 shutdown_connections 2024-09-21T01:20:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.947+0000 7f0b62800640 1 -- 172.21.15.27:0/545896162 wait complete. 2024-09-21T01:20:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.948+0000 7f0b62800640 1 Processor -- start 2024-09-21T01:20:42.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.948+0000 7f0b62800640 1 -- start start 2024-09-21T01:20:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b62800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c105d80 0x7f0b5c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b62800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b5c10b6c0 0x7f0b5c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b62800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b5c10c010 0x7f0b5c1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b62800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c115cd0 con 0x7f0b5c105d80 2024-09-21T01:20:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b62800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c115b50 con 0x7f0b5c10b6c0 2024-09-21T01:20:42.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b62800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c115e50 con 0x7f0b5c10c010 2024-09-21T01:20:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b5b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c105d80 0x7f0b5c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b5be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b5c10c010 0x7f0b5c1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b5b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c105d80 0x7f0b5c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55632/0 (socket says 172.21.15.27:55632) 2024-09-21T01:20:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b5b400640 1 -- 172.21.15.27:0/1277785943 learned_addr learned my addr 172.21.15.27:0/1277785943 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b5be00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b5c10c010 0x7f0b5c1a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51682/0 (socket says 172.21.15.27:51682) 2024-09-21T01:20:42.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.949+0000 7f0b5aa00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b5c10b6c0 0x7f0b5c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b5b400640 1 -- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b5c10c010 msgr2=0x7f0b5c1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b5b400640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b5c10c010 0x7f0b5c1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b5b400640 1 -- 172.21.15.27:0/1277785943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b5c10b6c0 msgr2=0x7f0b5c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b5b400640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0b5c10b6c0 0x7f0b5c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b5b400640 1 -- 172.21.15.27:0/1277785943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c1a25f0 con 0x7f0b5c105d80 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b5b400640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c105d80 0x7f0b5c19c680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00d950 tx=0x7f0b4c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c017030 con 0x7f0b5c105d80 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b4c004020 con 0x7f0b5c105d80 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c1a28e0 con 0x7f0b5c105d80 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.950+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c005560 con 0x7f0b5c105d80 2024-09-21T01:20:42.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.951+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c1a9040 con 0x7f0b5c105d80 2024-09-21T01:20:42.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.951+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b5c063b50 con 0x7f0b5c105d80 2024-09-21T01:20:42.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.954+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f0b4c009a30 con 0x7f0b5c105d80 2024-09-21T01:20:42.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.955+0000 7f0b53e00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b400b6670 0x7f0b400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.956+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f0b4c0e7930 con 0x7f0b5c105d80 2024-09-21T01:20:42.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.956+0000 7f0b5aa00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b400b6670 0x7f0b400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.956+0000 7f0b53e00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] conn(0x7f0b400c1b80 0x7f0b400c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.956+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 --> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b4c022e50 con 0x7f0b400c1b80 2024-09-21T01:20:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.956+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0b4c0e7d80 con 0x7f0b5c105d80 2024-09-21T01:20:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.957+0000 7f0b5be00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] conn(0x7f0b400c1b80 0x7f0b400c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.957+0000 7f0b5aa00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b400b6670 0x7f0b400b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c10b160 tx=0x7f0b48009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:42.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.957+0000 7f0b5be00640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] conn(0x7f0b400c1b80 0x7f0b400c4070 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:42.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.958+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== osd.2 v2:172.21.15.98:6800/2706735365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b4c022e50 con 0x7f0b400c1b80 2024-09-21T01:20:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.973+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 --> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b5c063f80 con 0x7f0b400c1b80 2024-09-21T01:20:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.973+0000 7f0b53e00640 1 -- 172.21.15.27:0/1277785943 <== osd.2 v2:172.21.15.98:6800/2706735365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b5c063f80 con 0x7f0b400c1b80 2024-09-21T01:20:42.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.973+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] conn(0x7f0b400c1b80 msgr2=0x7f0b400c4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.973+0000 7f0b62800640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] conn(0x7f0b400c1b80 0x7f0b400c4070 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:42.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b400b6670 msgr2=0x7f0b400b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f0b400b6670 0x7f0b400b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c10b160 tx=0x7f0b48009260 comp rx=0 tx=0).stop 2024-09-21T01:20:42.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c105d80 msgr2=0x7f0b5c19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:42.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 --2- 172.21.15.27:0/1277785943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b5c105d80 0x7f0b5c19c680 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00d950 tx=0x7f0b4c00dd50 comp rx=0 tx=0).stop 2024-09-21T01:20:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b5be00640 1 -- 172.21.15.27:0/1277785943 reap_dead start 2024-09-21T01:20:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 shutdown_connections 2024-09-21T01:20:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 >> 172.21.15.27:0/1277785943 conn(0x7f0b5c101810 msgr2=0x7f0b5c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 shutdown_connections 2024-09-21T01:20:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.974+0000 7f0b62800640 1 -- 172.21.15.27:0/1277785943 wait complete. 2024-09-21T01:20:43.274 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:43 smithi027 ceph-mon[30375]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T01:20:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:43 smithi089 ceph-mon[34486]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T01:20:43.549 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:43 smithi098 ceph-mon[34372]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 -- 172.21.15.27:0/2154977607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 msgr2=0x7f536810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 --2- 172.21.15.27:0/2154977607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 0x7f536810bb00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5354009900 tx=0x7f535402a400 comp rx=0 tx=0).stop 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 -- 172.21.15.27:0/2154977607 shutdown_connections 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 --2- 172.21.15.27:0/2154977607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536810c040 0x7f5368112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 --2- 172.21.15.27:0/2154977607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 0x7f536810bb00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 --2- 172.21.15.27:0/2154977607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 0x7f53681061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 -- 172.21.15.27:0/2154977607 >> 172.21.15.27:0/2154977607 conn(0x7f53681017e0 msgr2=0x7f5368103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 -- 172.21.15.27:0/2154977607 shutdown_connections 2024-09-21T01:20:43.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.177+0000 7f536ee00640 1 -- 172.21.15.27:0/2154977607 wait complete. 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.178+0000 7f536ee00640 1 Processor -- start 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 -- start start 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 0x7f536819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 0x7f536819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536810c040 0x7f53681a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5368115d50 con 0x7f536810b6f0 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5368115bd0 con 0x7f536810c040 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ee00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5368115ed0 con 0x7f5368105db0 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f5367e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 0x7f536819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f5367e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 0x7f536819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51672/0 (socket says 172.21.15.27:51672) 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f5367e00640 1 -- 172.21.15.27:0/1607731130 learned_addr learned my addr 172.21.15.27:0/1607731130 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f536ce00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536810c040 0x7f53681a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.179+0000 7f5367400640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 0x7f536819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5367e00640 1 -- 172.21.15.27:0/1607731130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536810c040 msgr2=0x7f53681a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:43.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5367e00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f536810c040 0x7f53681a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5367e00640 1 -- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 msgr2=0x7f536819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5367e00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f536810b6f0 0x7f536819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5367e00640 1 -- 172.21.15.27:0/1607731130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681a2730 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5367e00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 0x7f536819c7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f535800d950 tx=0x7f535800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.180+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358017070 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.181+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5358009c30 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.181+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53681a2a20 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.181+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535800f620 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.181+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a9180 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.181+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5368063b50 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.185+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f5358010460 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.186+0000 7f5364c00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f534c0b6670 0x7f534c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.187+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f53580e7910 con 0x7f5368105db0 2024-09-21T01:20:43.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.187+0000 7f5367400640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f534c0b6670 0x7f534c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.187+0000 7f5364c00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] conn(0x7f534c0c1b80 0x7f534c0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.187+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 --> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5358015e80 con 0x7f534c0c1b80 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.187+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f53580e7cc0 con 0x7f5368105db0 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.187+0000 7f536ce00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] conn(0x7f534c0c1b80 0x7f534c0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.188+0000 7f5367400640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f534c0b6670 0x7f534c0b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f535402a9f0 tx=0x7f5354005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.188+0000 7f536ce00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] conn(0x7f534c0c1b80 0x7f534c0c4070 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.189+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== osd.5 v2:172.21.15.98:6808/3161787475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5358015e80 con 0x7f534c0c1b80 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.204+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 --> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5368063fd0 con 0x7f534c0c1b80 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.204+0000 7f5364c00640 1 -- 172.21.15.27:0/1607731130 <== osd.5 v2:172.21.15.98:6808/3161787475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5368063fd0 con 0x7f534c0c1b80 2024-09-21T01:20:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] conn(0x7f534c0c1b80 msgr2=0x7f534c0c4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] conn(0x7f534c0c1b80 0x7f534c0c4070 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f534c0b6670 msgr2=0x7f534c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f534c0b6670 0x7f534c0b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f535402a9f0 tx=0x7f5354005a20 comp rx=0 tx=0).stop 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 msgr2=0x7f536819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 --2- 172.21.15.27:0/1607731130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5368105db0 0x7f536819c7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f535800d950 tx=0x7f535800de10 comp rx=0 tx=0).stop 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ce00640 1 -- 172.21.15.27:0/1607731130 reap_dead start 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 shutdown_connections 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 >> 172.21.15.27:0/1607731130 conn(0x7f53681017e0 msgr2=0x7f5368108d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 shutdown_connections 2024-09-21T01:20:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:42.205+0000 7f536ee00640 1 -- 172.21.15.27:0/1607731130 wait complete. 2024-09-21T01:20:44.150 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:44.151 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:44.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:44 smithi027 ceph-mon[30375]: pgmap v349: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6 op/s 2024-09-21T01:20:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:44 smithi089 ceph-mon[34486]: pgmap v349: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6 op/s 2024-09-21T01:20:44.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 -- 172.21.15.27:0/3744443191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c077940 msgr2=0x7f727c077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:44.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 --2- 172.21.15.27:0/3744443191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c077940 0x7f727c077d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c02a400 comp rx=0 tx=0).stop 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 -- 172.21.15.27:0/3744443191 shutdown_connections 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 --2- 172.21.15.27:0/3744443191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f727c114740 0x7f727c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 --2- 172.21.15.27:0/3744443191 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f727c078290 0x7f727c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 --2- 172.21.15.27:0/3744443191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c077940 0x7f727c077d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 -- 172.21.15.27:0/3744443191 >> 172.21.15.27:0/3744443191 conn(0x7f727c1017d0 msgr2=0x7f727c103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 -- 172.21.15.27:0/3744443191 shutdown_connections 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.614+0000 7f7281c00640 1 -- 172.21.15.27:0/3744443191 wait complete. 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.615+0000 7f7281c00640 1 Processor -- start 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 -- start start 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f727c077940 0x7f727c1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f727c078290 0x7f727c1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c114740 0x7f727c1a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f727c11a120 con 0x7f727c114740 2024-09-21T01:20:44.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f727c119fa0 con 0x7f727c078290 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f7281c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f727c11a2a0 con 0x7f727c077940 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f727aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f727c077940 0x7f727c1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f727b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c114740 0x7f727c1a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f727aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f727c077940 0x7f727c1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51700/0 (socket says 172.21.15.27:51700) 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.616+0000 7f727aa00640 1 -- 172.21.15.27:0/3145837624 learned_addr learned my addr 172.21.15.27:0/3145837624 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f7273e00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f727c078290 0x7f727c1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f727b400640 1 -- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f727c077940 msgr2=0x7f727c1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f727b400640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f727c077940 0x7f727c1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f727b400640 1 -- 172.21.15.27:0/3145837624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f727c078290 msgr2=0x7f727c1a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f727b400640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f727c078290 0x7f727c1a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f727b400640 1 -- 172.21.15.27:0/3145837624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c1a6a20 con 0x7f727c114740 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f727b400640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c114740 0x7f727c1a62f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7274009fd0 tx=0x7f727400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727400ce20 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274005020 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.617+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274010690 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.618+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1a6d10 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.618+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1ad470 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.619+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f727c063ad0 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.621+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f72740054d0 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f7273400640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72600b6670 0x7f72600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f72740e8160 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f727aa00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72600b6670 0x7f72600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f7273400640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] conn(0x7f72600c1b80 0x7f72600c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 --> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7274016e50 con 0x7f72600c1b80 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f72740e8510 con 0x7f727c114740 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.623+0000 7f7273e00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] conn(0x7f72600c1b80 0x7f72600c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.624+0000 7f727aa00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72600b6670 0x7f72600b8b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f726c0096c0 tx=0x7f726c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:44.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.624+0000 7f7273e00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] conn(0x7f72600c1b80 0x7f72600c4070 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:44.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.625+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== osd.8 v2:172.21.15.98:6816/1001086058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7274016e50 con 0x7f72600c1b80 2024-09-21T01:20:44.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.640+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 --> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f727c063fa0 con 0x7f72600c1b80 2024-09-21T01:20:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.641+0000 7f7273400640 1 -- 172.21.15.27:0/3145837624 <== osd.8 v2:172.21.15.98:6816/1001086058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f727c063fa0 con 0x7f72600c1b80 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] conn(0x7f72600c1b80 msgr2=0x7f72600c4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] conn(0x7f72600c1b80 0x7f72600c4070 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72600b6670 msgr2=0x7f72600b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f72600b6670 0x7f72600b8b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f726c0096c0 tx=0x7f726c005a20 comp rx=0 tx=0).stop 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c114740 msgr2=0x7f727c1a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 --2- 172.21.15.27:0/3145837624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f727c114740 0x7f727c1a62f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7274009fd0 tx=0x7f727400ef10 comp rx=0 tx=0).stop 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f727b400640 1 -- 172.21.15.27:0/3145837624 reap_dead start 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 shutdown_connections 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 >> 172.21.15.27:0/3145837624 conn(0x7f727c1017d0 msgr2=0x7f727c116800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 shutdown_connections 2024-09-21T01:20:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.642+0000 7f7281c00640 1 -- 172.21.15.27:0/3145837624 wait complete. 2024-09-21T01:20:44.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:44 smithi098 ceph-mon[34372]: pgmap v349: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6 op/s 2024-09-21T01:20:45.272 INFO:teuthology.orchestra.run.smithi027.stdout:158913790021 2024-09-21T01:20:45.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T01:20:45.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 -- 172.21.15.27:0/2378770029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510114570 msgr2=0x7f3510116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 --2- 172.21.15.27:0/2378770029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510114570 0x7f3510116990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3508009be0 tx=0x7f350802a6d0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 -- 172.21.15.27:0/2378770029 shutdown_connections 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 --2- 172.21.15.27:0/2378770029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510114570 0x7f3510116990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 --2- 172.21.15.27:0/2378770029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510078290 0x7f3510114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 --2- 172.21.15.27:0/2378770029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3510077940 0x7f3510077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 -- 172.21.15.27:0/2378770029 >> 172.21.15.27:0/2378770029 conn(0x7f3510101680 msgr2=0x7f3510103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 -- 172.21.15.27:0/2378770029 shutdown_connections 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.927+0000 7f350f200640 1 -- 172.21.15.27:0/2378770029 wait complete. 2024-09-21T01:20:45.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.928+0000 7f350f200640 1 Processor -- start 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.928+0000 7f350f200640 1 -- start start 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510077940 0x7f35101a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510078290 0x7f35101a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3510114570 0x7f35101a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3510119ed0 con 0x7f3510114570 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3510119d50 con 0x7f3510078290 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f351011a050 con 0x7f3510077940 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510078290 0x7f35101a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510077940 0x7f35101a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510077940 0x7f35101a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51714/0 (socket says 172.21.15.27:51714) 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 -- 172.21.15.27:0/1935304280 learned_addr learned my addr 172.21.15.27:0/1935304280 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350d400640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510078290 0x7f35101a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:43234/0 (socket says 172.21.15.27:43234) 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350e800640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3510114570 0x7f35101a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:45.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 -- 172.21.15.27:0/1935304280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510078290 msgr2=0x7f35101a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510078290 0x7f35101a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 -- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3510114570 msgr2=0x7f35101a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3510114570 0x7f35101a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.929+0000 7f350de00640 1 -- 172.21.15.27:0/1935304280 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35101a6870 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f350d400640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3510078290 0x7f35101a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f350de00640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510077940 0x7f35101a0900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f350000e9c0 tx=0x7f350000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350000cde0 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35101a6b60 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35000035f0 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500010690 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.930+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35101ad2c0 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.931+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3510063b50 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.934+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3500004000 con 0x7f3510077940 2024-09-21T01:20:45.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.936+0000 7f3506a00640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f34f40b65f0 0x7f34f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:45.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.936+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f35000e7fb0 con 0x7f3510077940 2024-09-21T01:20:45.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.936+0000 7f350d400640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f34f40b65f0 0x7f34f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:45.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.936+0000 7f3506a00640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] conn(0x7f34f40c1af0 0x7f34f40c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:45.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.936+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 --> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3500016ec0 con 0x7f34f40c1af0 2024-09-21T01:20:45.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.936+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f35000e8360 con 0x7f3510077940 2024-09-21T01:20:45.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.937+0000 7f350e800640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] conn(0x7f34f40c1af0 0x7f34f40c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.937+0000 7f350d400640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f34f40b65f0 0x7f34f40b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f351007b220 tx=0x7f34fc005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.937+0000 7f350e800640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] conn(0x7f34f40c1af0 0x7f34f40c3fe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.938+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== osd.0 v2:172.21.15.27:6802/3167750643 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3500016ec0 con 0x7f34f40c1af0 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.952+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 --> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3510063f80 con 0x7f34f40c1af0 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f3506a00640 1 -- 172.21.15.27:0/1935304280 <== osd.0 v2:172.21.15.27:6802/3167750643 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3510063f80 con 0x7f34f40c1af0 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] conn(0x7f34f40c1af0 msgr2=0x7f34f40c3fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f350f200640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] conn(0x7f34f40c1af0 0x7f34f40c3fe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f34f40b65f0 msgr2=0x7f34f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f350f200640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f34f40b65f0 0x7f34f40b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f351007b220 tx=0x7f34fc005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510077940 msgr2=0x7f35101a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.953+0000 7f350f200640 1 --2- 172.21.15.27:0/1935304280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3510077940 0x7f35101a0900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f350000e9c0 tx=0x7f350000ee80 comp rx=0 tx=0).stop 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.954+0000 7f350e800640 1 -- 172.21.15.27:0/1935304280 reap_dead start 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.954+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 shutdown_connections 2024-09-21T01:20:45.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.954+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 >> 172.21.15.27:0/1935304280 conn(0x7f3510101680 msgr2=0x7f3510116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:45.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.954+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 shutdown_connections 2024-09-21T01:20:45.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:44.954+0000 7f350f200640 1 -- 172.21.15.27:0/1935304280 wait complete. 2024-09-21T01:20:46.359 INFO:teuthology.orchestra.run.smithi027.stdout:38654705761 2024-09-21T01:20:46.360 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T01:20:46.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:46 smithi089 ceph-mon[34486]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3 op/s; 16 B/s, 0 objects/s recovering 2024-09-21T01:20:46.646 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:46.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:46 smithi098 ceph-mon[34372]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3 op/s; 16 B/s, 0 objects/s recovering 2024-09-21T01:20:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:46 smithi027 ceph-mon[30375]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3 op/s; 16 B/s, 0 objects/s recovering 2024-09-21T01:20:47.246 INFO:teuthology.orchestra.run.smithi027.stdout:81604378709 2024-09-21T01:20:47.246 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T01:20:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 -- 172.21.15.27:0/1300674670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d010b500 msgr2=0x7fb5d010b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/1300674670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d010b500 0x7fb5d010b910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009900 tx=0x7fb5bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 -- 172.21.15.27:0/1300674670 shutdown_connections 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/1300674670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d010be50 0x7fb5d0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/1300674670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d010b500 0x7fb5d010b910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/1300674670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d0105bc0 0x7fb5d0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 -- 172.21.15.27:0/1300674670 >> 172.21.15.27:0/1300674670 conn(0x7fb5d0101670 msgr2=0x7fb5d0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 -- 172.21.15.27:0/1300674670 shutdown_connections 2024-09-21T01:20:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.869+0000 7fb5d4e00640 1 -- 172.21.15.27:0/1300674670 wait complete. 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.870+0000 7fb5d4e00640 1 Processor -- start 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 -- start start 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d0105bc0 0x7fb5d019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d010b500 0x7fb5d019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d010be50 0x7fb5d01a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0115b10 con 0x7fb5d0105bc0 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0115990 con 0x7fb5d010b500 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5d4e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0115c90 con 0x7fb5d010be50 2024-09-21T01:20:47.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d0105bc0 0x7fb5d019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:47.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d010b500 0x7fb5d019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:47.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5cea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d010b500 0x7fb5d019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:44470/0 (socket says 172.21.15.27:44470) 2024-09-21T01:20:47.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5cea00640 1 -- 172.21.15.27:0/3832420743 learned_addr learned my addr 172.21.15.27:0/3832420743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:47.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.871+0000 7fb5cf400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d0105bc0 0x7fb5d019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34456/0 (socket says 172.21.15.27:34456) 2024-09-21T01:20:47.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cfe00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d010be50 0x7fb5d01a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cea00640 1 -- 172.21.15.27:0/3832420743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d010be50 msgr2=0x7fb5d01a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cea00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d010be50 0x7fb5d01a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cea00640 1 -- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d0105bc0 msgr2=0x7fb5d019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cea00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d0105bc0 0x7fb5d019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cea00640 1 -- 172.21.15.27:0/3832420743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d01a2470 con 0x7fb5d010b500 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.872+0000 7fb5cea00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d010b500 0x7fb5d019caa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc02a910 tx=0x7fb5bc031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.873+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc03f030 con 0x7fb5d010b500 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.873+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5bc004490 con 0x7fb5d010b500 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.873+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d01a2700 con 0x7fb5d010b500 2024-09-21T01:20:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.873+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc035070 con 0x7fb5d010b500 2024-09-21T01:20:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.873+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01a8ec0 con 0x7fb5d010b500 2024-09-21T01:20:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.874+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5d0063b50 con 0x7fb5d010b500 2024-09-21T01:20:47.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.877+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fb5bc042030 con 0x7fb5d010b500 2024-09-21T01:20:47.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.878+0000 7fb5c7e00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb5b40b65f0 0x7fb5b40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.878+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb5bc108110 con 0x7fb5d010b500 2024-09-21T01:20:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.878+0000 7fb5cf400640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb5b40b65f0 0x7fb5b40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.879+0000 7fb5c7e00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] conn(0x7fb5b40c1b00 0x7fb5b40c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.879+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 --> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5bc04eea0 con 0x7fb5b40c1b00 2024-09-21T01:20:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.879+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb5bc1084c0 con 0x7fb5d010b500 2024-09-21T01:20:47.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.879+0000 7fb5cfe00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] conn(0x7fb5b40c1b00 0x7fb5b40c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:47.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.879+0000 7fb5cf400640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb5b40b65f0 0x7fb5b40b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009bd0 tx=0x7fb5c0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:47.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.880+0000 7fb5cfe00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] conn(0x7fb5b40c1b00 0x7fb5b40c3ff0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:47.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.881+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== osd.9 v2:172.21.15.27:6826/1133805797 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5bc04eea0 con 0x7fb5b40c1b00 2024-09-21T01:20:47.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.895+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 --> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5d0063fd0 con 0x7fb5b40c1b00 2024-09-21T01:20:47.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5c7e00640 1 -- 172.21.15.27:0/3832420743 <== osd.9 v2:172.21.15.27:6826/1133805797 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb5d0063fd0 con 0x7fb5b40c1b00 2024-09-21T01:20:47.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] conn(0x7fb5b40c1b00 msgr2=0x7fb5b40c3ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] conn(0x7fb5b40c1b00 0x7fb5b40c3ff0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb5b40b65f0 msgr2=0x7fb5b40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb5b40b65f0 0x7fb5b40b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009bd0 tx=0x7fb5c0005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d010b500 msgr2=0x7fb5d019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 --2- 172.21.15.27:0/3832420743 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb5d010b500 0x7fb5d019caa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc02a910 tx=0x7fb5bc031a20 comp rx=0 tx=0).stop 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5cfe00640 1 -- 172.21.15.27:0/3832420743 reap_dead start 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 shutdown_connections 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 >> 172.21.15.27:0/3832420743 conn(0x7fb5d0101670 msgr2=0x7fb5d0108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:47.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 shutdown_connections 2024-09-21T01:20:47.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:47.896+0000 7fb5d4e00640 1 -- 172.21.15.27:0/3832420743 wait complete. 2024-09-21T01:20:48.498 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:48 smithi027 ceph-mon[30375]: pgmap v351: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-09-21T01:20:48.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:48 smithi089 ceph-mon[34486]: pgmap v351: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-09-21T01:20:48.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:48 smithi098 ceph-mon[34372]: pgmap v351: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-09-21T01:20:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.826+0000 7f69f6a00640 1 -- 172.21.15.27:0/1562355412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 msgr2=0x7f69f00761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.826+0000 7f69f6a00640 1 --2- 172.21.15.27:0/1562355412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 0x7f69f00761e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009900 tx=0x7f69dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 -- 172.21.15.27:0/1562355412 shutdown_connections 2024-09-21T01:20:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 --2- 172.21.15.27:0/1562355412 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f0114260 0x7f69f0116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 --2- 172.21.15.27:0/1562355412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 0x7f69f00761e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 --2- 172.21.15.27:0/1562355412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 0x7f69f00757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 -- 172.21.15.27:0/1562355412 >> 172.21.15.27:0/1562355412 conn(0x7f69f00ff3e0 msgr2=0x7f69f0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 -- 172.21.15.27:0/1562355412 shutdown_connections 2024-09-21T01:20:48.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.827+0000 7f69f6a00640 1 -- 172.21.15.27:0/1562355412 wait complete. 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 Processor -- start 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 -- start start 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 0x7f69f01a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 0x7f69f01a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f0114260 0x7f69f01a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0119ef0 con 0x7f69f0075dd0 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0119d70 con 0x7f69f0114260 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.828+0000 7f69f6a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f69f011a070 con 0x7f69f0077380 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 0x7f69f01a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 0x7f69f01a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60754/0 (socket says 172.21.15.27:60754) 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 -- 172.21.15.27:0/2053430827 learned_addr learned my addr 172.21.15.27:0/2053430827 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f5600640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 0x7f69f01a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f6000640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f0114260 0x7f69f01a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 -- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f0114260 msgr2=0x7f69f01a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f69f0114260 0x7f69f01a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 -- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 msgr2=0x7f69f01a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69f0075dd0 0x7f69f01a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.829+0000 7f69f4c00640 1 -- 172.21.15.27:0/2053430827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f01a68f0 con 0x7f69f0077380 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.830+0000 7f69f4c00640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 0x7f69f01a0f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005ab0 tx=0x7f69dc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.830+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc03f070 con 0x7f69f0077380 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.830+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01a6b80 con 0x7f69f0077380 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.830+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69dc031b50 con 0x7f69f0077380 2024-09-21T01:20:48.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.830+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc035030 con 0x7f69f0077380 2024-09-21T01:20:48.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.830+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f01ad340 con 0x7f69f0077380 2024-09-21T01:20:48.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.831+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69f0063b50 con 0x7f69f0077380 2024-09-21T01:20:48.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.834+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f69dc042070 con 0x7f69f0077380 2024-09-21T01:20:48.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.835+0000 7f69ee000640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69d80b65f0 0x7f69d80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f69dc108370 con 0x7f69f0077380 2024-09-21T01:20:48.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69f5600640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69d80b65f0 0x7f69d80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69ee000640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] conn(0x7f69d80c1b00 0x7f69d80c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 --> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69d80c46c0 con 0x7f69d80c1b00 2024-09-21T01:20:48.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f69dc1087b0 con 0x7f69f0077380 2024-09-21T01:20:48.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69f6000640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] conn(0x7f69d80c1b00 0x7f69d80c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.836+0000 7f69f5600640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69d80b65f0 0x7f69d80b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009bd0 tx=0x7f69e4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:48.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.837+0000 7f69f6000640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] conn(0x7f69d80c1b00 0x7f69d80c3ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:48.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.838+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== osd.1 v2:172.21.15.89:6800/1010396309 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69d80c46c0 con 0x7f69d80c1b00 2024-09-21T01:20:48.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.853+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 --> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69f0063f80 con 0x7f69d80c1b00 2024-09-21T01:20:48.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.853+0000 7f69ee000640 1 -- 172.21.15.27:0/2053430827 <== osd.1 v2:172.21.15.89:6800/1010396309 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f69f0063f80 con 0x7f69d80c1b00 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] conn(0x7f69d80c1b00 msgr2=0x7f69d80c3ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] conn(0x7f69d80c1b00 0x7f69d80c3ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69d80b65f0 msgr2=0x7f69d80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69d80b65f0 0x7f69d80b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009bd0 tx=0x7f69e4005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 msgr2=0x7f69f01a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 --2- 172.21.15.27:0/2053430827 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69f0077380 0x7f69f01a0f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005ab0 tx=0x7f69dc003b90 comp rx=0 tx=0).stop 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6000640 1 -- 172.21.15.27:0/2053430827 reap_dead start 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 shutdown_connections 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 >> 172.21.15.27:0/2053430827 conn(0x7f69f00ff3e0 msgr2=0x7f69f0110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:48.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 shutdown_connections 2024-09-21T01:20:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.854+0000 7f69f6a00640 1 -- 172.21.15.27:0/2053430827 wait complete. 2024-09-21T01:20:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 -- 172.21.15.27:0/3267361898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268105110 msgr2=0x7f826810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 --2- 172.21.15.27:0/3267361898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268105110 0x7f826810d660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8254009900 tx=0x7f825402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 -- 172.21.15.27:0/3267361898 shutdown_connections 2024-09-21T01:20:48.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 --2- 172.21.15.27:0/3267361898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f826810dc20 0x7f8268110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 --2- 172.21.15.27:0/3267361898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268105110 0x7f826810d660 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 --2- 172.21.15.27:0/3267361898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82681047c0 0x7f8268104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 -- 172.21.15.27:0/3267361898 >> 172.21.15.27:0/3267361898 conn(0x7f82680fe2c0 msgr2=0x7f8268100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 -- 172.21.15.27:0/3267361898 shutdown_connections 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.696+0000 7f826d800640 1 -- 172.21.15.27:0/3267361898 wait complete. 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.697+0000 7f826d800640 1 Processor -- start 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 -- start start 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82681047c0 0x7f826819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8268105110 0x7f826819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f826810dc20 0x7f82681a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8268113880 con 0x7f82681047c0 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8268113700 con 0x7f8268105110 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f826d800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8268113a00 con 0x7f826810dc20 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.698+0000 7f8267e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82681047c0 0x7f826819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82681047c0 0x7f826819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34480/0 (socket says 172.21.15.27:34480) 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267e00640 1 -- 172.21.15.27:0/2908566348 learned_addr learned my addr 172.21.15.27:0/2908566348 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8268105110 0x7f826819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f826ce00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f826810dc20 0x7f82681a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 -- 172.21.15.27:0/2908566348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f826810dc20 msgr2=0x7f82681a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f826810dc20 0x7f82681a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 -- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82681047c0 msgr2=0x7f826819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82681047c0 0x7f826819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 -- 172.21.15.27:0/2908566348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681a2490 con 0x7f8268105110 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267e00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82681047c0 0x7f826819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.699+0000 7f8267400640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8268105110 0x7f826819cac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f825402a910 tx=0x7f8254031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.700+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825403f070 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.700+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681a2720 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.700+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8254004490 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.700+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254035070 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.700+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681a8ee0 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.701+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f826810d760 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.704+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f825403b460 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.705+0000 7f8264c00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f824c0b65f0 0x7f824c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.706+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f82541084f0 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.706+0000 7f8267e00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f824c0b65f0 0x7f824c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.706+0000 7f8264c00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] conn(0x7f824c0c1b00 0x7f824c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.706+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 --> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8254040ec0 con 0x7f824c0c1b00 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.706+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f82541088a0 con 0x7f8268105110 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.706+0000 7f826ce00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] conn(0x7f824c0c1b00 0x7f824c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.707+0000 7f8267e00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f824c0b65f0 0x7f824c0b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82580095f0 tx=0x7f8258005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.707+0000 7f826ce00640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] conn(0x7f824c0c1b00 0x7f824c0c3ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.708+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== osd.7 v2:172.21.15.89:6816/2410726915 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8254040ec0 con 0x7f824c0c1b00 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.722+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 --> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f826806b560 con 0x7f824c0c1b00 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.723+0000 7f8264c00640 1 -- 172.21.15.27:0/2908566348 <== osd.7 v2:172.21.15.89:6816/2410726915 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f826806b560 con 0x7f824c0c1b00 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] conn(0x7f824c0c1b00 msgr2=0x7f824c0c3ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] conn(0x7f824c0c1b00 0x7f824c0c3ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f824c0b65f0 msgr2=0x7f824c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f824c0b65f0 0x7f824c0b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82580095f0 tx=0x7f8258005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8268105110 msgr2=0x7f826819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 --2- 172.21.15.27:0/2908566348 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8268105110 0x7f826819cac0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f825402a910 tx=0x7f8254031a20 comp rx=0 tx=0).stop 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826ce00640 1 -- 172.21.15.27:0/2908566348 reap_dead start 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 shutdown_connections 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 >> 172.21.15.27:0/2908566348 conn(0x7f82680fe2c0 msgr2=0x7f826810cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 shutdown_connections 2024-09-21T01:20:48.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.724+0000 7f826d800640 1 -- 172.21.15.27:0/2908566348 wait complete. 2024-09-21T01:20:49.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:49 smithi089 ceph-mon[34486]: pgmap v352: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-09-21T01:20:49.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:49 smithi098 ceph-mon[34372]: pgmap v352: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-09-21T01:20:49.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:49 smithi027 ceph-mon[30375]: pgmap v352: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2 op/s; 8.2 KiB/s, 1 objects/s recovering 2024-09-21T01:20:49.881 INFO:teuthology.orchestra.run.smithi027.stdout:206158430265 2024-09-21T01:20:49.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T01:20:50.727 INFO:teuthology.orchestra.run.smithi027.stdout:137438953546 2024-09-21T01:20:50.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T01:20:51.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.002+0000 7f398a600640 1 -- 172.21.15.27:0/3301990573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984102440 msgr2=0x7f3984102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.002+0000 7f398a600640 1 --2- 172.21.15.27:0/3301990573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984102440 0x7f3984102850 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3978009860 tx=0x7f397802a3d0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 -- 172.21.15.27:0/3301990573 shutdown_connections 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 --2- 172.21.15.27:0/3301990573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f398410fff0 0x7f3984112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 --2- 172.21.15.27:0/3301990573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3984102d90 0x7f398410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 --2- 172.21.15.27:0/3301990573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984102440 0x7f3984102850 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 -- 172.21.15.27:0/3301990573 >> 172.21.15.27:0/3301990573 conn(0x7f39840fe2c0 msgr2=0x7f3984100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 -- 172.21.15.27:0/3301990573 shutdown_connections 2024-09-21T01:20:51.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.003+0000 7f398a600640 1 -- 172.21.15.27:0/3301990573 wait complete. 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 Processor -- start 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 -- start start 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3984102440 0x7f398419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3984102d90 0x7f398419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398410fff0 0x7f39841a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3984115ae0 con 0x7f398410fff0 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3984115960 con 0x7f3984102440 2024-09-21T01:20:51.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.004+0000 7f398a600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3984115c60 con 0x7f3984102d90 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398410fff0 0x7f39841a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3984102440 0x7f398419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398410fff0 0x7f39841a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34530/0 (socket says 172.21.15.27:34530) 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989c00640 1 -- 172.21.15.27:0/3140796959 learned_addr learned my addr 172.21.15.27:0/3140796959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3983e00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3984102d90 0x7f398419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989200640 1 -- 172.21.15.27:0/3140796959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3984102d90 msgr2=0x7f398419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989200640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3984102d90 0x7f398419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989200640 1 -- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398410fff0 msgr2=0x7f39841a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:51.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989200640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398410fff0 0x7f39841a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989200640 1 -- 172.21.15.27:0/3140796959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39841a2420 con 0x7f3984102440 2024-09-21T01:20:51.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3989c00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f398410fff0 0x7f39841a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:51.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.005+0000 7f3983e00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3984102d90 0x7f398419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:51.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.006+0000 7f3989200640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3984102440 0x7f398419c4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3978005940 tx=0x7f3978030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.006+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978040070 con 0x7f3984102440 2024-09-21T01:20:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.006+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39841a26b0 con 0x7f3984102440 2024-09-21T01:20:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.006+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3978005570 con 0x7f3984102440 2024-09-21T01:20:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.006+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780046c0 con 0x7f3984102440 2024-09-21T01:20:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.006+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39841a8e70 con 0x7f3984102440 2024-09-21T01:20:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.007+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3984063b10 con 0x7f3984102440 2024-09-21T01:20:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.010+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3978043070 con 0x7f3984102440 2024-09-21T01:20:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.012+0000 7f3981600640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f396c0b65f0 0x7f396c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.012+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f39781088c0 con 0x7f3984102440 2024-09-21T01:20:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.012+0000 7f3983e00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f396c0b65f0 0x7f396c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.012+0000 7f3981600640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] conn(0x7f396c0c1b00 0x7f396c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.012+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 --> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3978031c50 con 0x7f396c0c1b00 2024-09-21T01:20:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.012+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3978108c70 con 0x7f3984102440 2024-09-21T01:20:51.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.013+0000 7f3989c00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] conn(0x7f396c0c1b00 0x7f396c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:51.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.013+0000 7f3983e00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f396c0b65f0 0x7f396c0b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f39700096c0 tx=0x7f3970009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:51.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.013+0000 7f3989c00640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] conn(0x7f396c0c1b00 0x7f396c0c3ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:51.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.014+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== osd.3 v2:172.21.15.27:6810/4289720168 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3978031c50 con 0x7f396c0c1b00 2024-09-21T01:20:51.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.029+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 --> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3984063f40 con 0x7f396c0c1b00 2024-09-21T01:20:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.029+0000 7f3981600640 1 -- 172.21.15.27:0/3140796959 <== osd.3 v2:172.21.15.27:6810/4289720168 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3984063f40 con 0x7f396c0c1b00 2024-09-21T01:20:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.029+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] conn(0x7f396c0c1b00 msgr2=0x7f396c0c3ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:51.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.029+0000 7f398a600640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] conn(0x7f396c0c1b00 0x7f396c0c3ff0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f396c0b65f0 msgr2=0x7f396c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f396c0b65f0 0x7f396c0b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f39700096c0 tx=0x7f3970009260 comp rx=0 tx=0).stop 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3984102440 msgr2=0x7f398419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 --2- 172.21.15.27:0/3140796959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3984102440 0x7f398419c4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3978005940 tx=0x7f3978030fc0 comp rx=0 tx=0).stop 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f3989c00640 1 -- 172.21.15.27:0/3140796959 reap_dead start 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 shutdown_connections 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 >> 172.21.15.27:0/3140796959 conn(0x7f39840fe2c0 msgr2=0x7f39841010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 shutdown_connections 2024-09-21T01:20:51.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:51.030+0000 7f398a600640 1 -- 172.21.15.27:0/3140796959 wait complete. 2024-09-21T01:20:51.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:51 smithi027 ceph-mon[30375]: pgmap v353: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6 op/s; 7.1 KiB/s, 1 objects/s recovering 2024-09-21T01:20:51.726 INFO:teuthology.orchestra.run.smithi027.stdout:231928234034 2024-09-21T01:20:51.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T01:20:51.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:51 smithi098 ceph-mon[34372]: pgmap v353: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6 op/s; 7.1 KiB/s, 1 objects/s recovering 2024-09-21T01:20:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:51 smithi089 ceph-mon[34486]: pgmap v353: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6 op/s; 7.1 KiB/s, 1 objects/s recovering 2024-09-21T01:20:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 -- 172.21.15.27:0/2957927865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 msgr2=0x7fde1010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 --2- 172.21.15.27:0/2957927865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1010bad0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009900 tx=0x7fddfc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 -- 172.21.15.27:0/2957927865 shutdown_connections 2024-09-21T01:20:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 --2- 172.21.15.27:0/2957927865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde1010c010 0x7fde10112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 --2- 172.21.15.27:0/2957927865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1010bad0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 --2- 172.21.15.27:0/2957927865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde10105d80 0x7fde10106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 -- 172.21.15.27:0/2957927865 >> 172.21.15.27:0/2957927865 conn(0x7fde10101810 msgr2=0x7fde10103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 -- 172.21.15.27:0/2957927865 shutdown_connections 2024-09-21T01:20:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.384+0000 7fde16600640 1 -- 172.21.15.27:0/2957927865 wait complete. 2024-09-21T01:20:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.385+0000 7fde16600640 1 Processor -- start 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 -- start start 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde10105d80 0x7fde1019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde1010c010 0x7fde101a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde10115cd0 con 0x7fde1010c010 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde10115b50 con 0x7fde1010b6c0 2024-09-21T01:20:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde16600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde10115e50 con 0x7fde10105d80 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde0fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde1010c010 0x7fde101a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde0ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:44592/0 (socket says 172.21.15.27:44592) 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde0ea00640 1 -- 172.21.15.27:0/4191418013 learned_addr learned my addr 172.21.15.27:0/4191418013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.386+0000 7fde0f400640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde10105d80 0x7fde1019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde0ea00640 1 -- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde10105d80 msgr2=0x7fde1019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde0ea00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde10105d80 0x7fde1019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde0ea00640 1 -- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde1010c010 msgr2=0x7fde101a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde0ea00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde1010c010 0x7fde101a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde0ea00640 1 -- 172.21.15.27:0/4191418013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde101a2610 con 0x7fde1010b6c0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde0ea00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1019cc40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fddfc02a910 tx=0x7fddfc031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.387+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc03f030 con 0x7fde1010b6c0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.388+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde101a28a0 con 0x7fde1010b6c0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.388+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddfc004490 con 0x7fde1010b6c0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.388+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc035070 con 0x7fde1010b6c0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.388+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101a9060 con 0x7fde1010b6c0 2024-09-21T01:20:52.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.389+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde10063b50 con 0x7fde1010b6c0 2024-09-21T01:20:52.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.391+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fddfc042030 con 0x7fde1010b6c0 2024-09-21T01:20:52.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.393+0000 7fde07e00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fddf40b6670 0x7fddf40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.393+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fddfc1080b0 con 0x7fde1010b6c0 2024-09-21T01:20:52.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.393+0000 7fde0f400640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fddf40b6670 0x7fddf40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.393+0000 7fde07e00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] conn(0x7fddf40c1b80 0x7fddf40c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.393+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 --> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddf40c4740 con 0x7fddf40c1b80 2024-09-21T01:20:52.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.393+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fddfc108460 con 0x7fde1010b6c0 2024-09-21T01:20:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.394+0000 7fde0fe00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] conn(0x7fddf40c1b80 0x7fddf40c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.394+0000 7fde0f400640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fddf40b6670 0x7fddf40b8b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fde00009bd0 tx=0x7fde00005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:52.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.394+0000 7fde0fe00640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] conn(0x7fddf40c1b80 0x7fddf40c4070 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:52.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.395+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== osd.11 v2:172.21.15.98:6824/165111690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddf40c4740 con 0x7fddf40c1b80 2024-09-21T01:20:52.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.409+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 --> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fde10063f50 con 0x7fddf40c1b80 2024-09-21T01:20:52.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.410+0000 7fde07e00640 1 -- 172.21.15.27:0/4191418013 <== osd.11 v2:172.21.15.98:6824/165111690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fde10063f50 con 0x7fddf40c1b80 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] conn(0x7fddf40c1b80 msgr2=0x7fddf40c4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] conn(0x7fddf40c1b80 0x7fddf40c4070 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fddf40b6670 msgr2=0x7fddf40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fddf40b6670 0x7fddf40b8b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fde00009bd0 tx=0x7fde00005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 msgr2=0x7fde1019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 --2- 172.21.15.27:0/4191418013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde1010b6c0 0x7fde1019cc40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fddfc02a910 tx=0x7fddfc031a20 comp rx=0 tx=0).stop 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde0fe00640 1 -- 172.21.15.27:0/4191418013 reap_dead start 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 shutdown_connections 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 >> 172.21.15.27:0/4191418013 conn(0x7fde10101810 msgr2=0x7fde10103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 shutdown_connections 2024-09-21T01:20:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:52.411+0000 7fde16600640 1 -- 172.21.15.27:0/4191418013 wait complete. 2024-09-21T01:20:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 -- 172.21.15.27:0/718681175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 msgr2=0x7f7e28116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 --2- 172.21.15.27:0/718681175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 0x7f7e28116750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009ca0 tx=0x7f7e2002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 -- 172.21.15.27:0/718681175 shutdown_connections 2024-09-21T01:20:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 --2- 172.21.15.27:0/718681175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 0x7f7e28116750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 --2- 172.21.15.27:0/718681175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 0x7f7e280761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 --2- 172.21.15.27:0/718681175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e28077380 0x7f7e280757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 -- 172.21.15.27:0/718681175 >> 172.21.15.27:0/718681175 conn(0x7f7e280ff440 msgr2=0x7f7e28101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 -- 172.21.15.27:0/718681175 shutdown_connections 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.561+0000 7f7e2da00640 1 -- 172.21.15.27:0/718681175 wait complete. 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.562+0000 7f7e2da00640 1 Processor -- start 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.562+0000 7f7e2da00640 1 -- start start 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 0x7f7e281a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e28077380 0x7f7e281a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 0x7f7e281a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28119f10 con 0x7f7e28114280 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28119d90 con 0x7f7e28075dd0 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2da00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2811a090 con 0x7f7e28077380 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 0x7f7e281a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e27e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 0x7f7e281a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:44492/0 (socket says 172.21.15.27:44492) 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e27400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e28077380 0x7f7e281a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e27e00640 1 -- 172.21.15.27:0/553509887 learned_addr learned my addr 172.21.15.27:0/553509887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.563+0000 7f7e2d000640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 0x7f7e281a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e27e00640 1 -- 172.21.15.27:0/553509887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e28077380 msgr2=0x7f7e281a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e27e00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e28077380 0x7f7e281a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e27e00640 1 -- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 msgr2=0x7f7e281a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e27e00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e28114280 0x7f7e281a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e27e00640 1 -- 172.21.15.27:0/553509887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e281a6830 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e27e00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 0x7f7e281a08c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800e9f0 tx=0x7f7e1800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800cd80 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e281a6b20 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.564+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e180054d0 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.565+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18010690 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.565+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281ad280 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.565+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e28063b10 con 0x7f7e28075dd0 2024-09-21T01:20:52.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.568+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f7e18004e30 con 0x7f7e28075dd0 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.570+0000 7f7e24c00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7e0c0b65f0 0x7f7e0c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.570+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7e180e8340 con 0x7f7e28075dd0 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.570+0000 7f7e27400640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7e0c0b65f0 0x7f7e0c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.570+0000 7f7e24c00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] conn(0x7f7e0c0c1b00 0x7f7e0c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.570+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 --> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e18015e80 con 0x7f7e0c0c1b00 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.570+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7e180e86f0 con 0x7f7e28075dd0 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.571+0000 7f7e2d000640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] conn(0x7f7e0c0c1b00 0x7f7e0c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:52.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.571+0000 7f7e27400640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7e0c0b65f0 0x7f7e0c0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e28116020 tx=0x7f7e14005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.571+0000 7f7e2d000640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] conn(0x7f7e0c0c1b00 0x7f7e0c0c3ff0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.572+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== osd.6 v2:172.21.15.27:6818/1151323802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e18015e80 con 0x7f7e0c0c1b00 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.587+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 --> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7e28063f90 con 0x7f7e0c0c1b00 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.587+0000 7f7e24c00640 1 -- 172.21.15.27:0/553509887 <== osd.6 v2:172.21.15.27:6818/1151323802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7e28063f90 con 0x7f7e0c0c1b00 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] conn(0x7f7e0c0c1b00 msgr2=0x7f7e0c0c3ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] conn(0x7f7e0c0c1b00 0x7f7e0c0c3ff0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7e0c0b65f0 msgr2=0x7f7e0c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7e0c0b65f0 0x7f7e0c0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e28116020 tx=0x7f7e14005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 msgr2=0x7f7e281a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 --2- 172.21.15.27:0/553509887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7e28075dd0 0x7f7e281a08c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800e9f0 tx=0x7f7e1800edf0 comp rx=0 tx=0).stop 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2d000640 1 -- 172.21.15.27:0/553509887 reap_dead start 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 shutdown_connections 2024-09-21T01:20:52.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 >> 172.21.15.27:0/553509887 conn(0x7f7e280ff440 msgr2=0x7f7e28110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:52.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 shutdown_connections 2024-09-21T01:20:52.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:48.588+0000 7f7e2da00640 1 -- 172.21.15.27:0/553509887 wait complete. 2024-09-21T01:20:53.176 INFO:teuthology.orchestra.run.smithi027.stdout:184683593791 2024-09-21T01:20:53.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T01:20:53.474 INFO:teuthology.orchestra.run.smithi027.stdout:60129542235 2024-09-21T01:20:53.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T01:20:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:53 smithi098 ceph-mon[34372]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 6.4 KiB/s, 1 objects/s recovering 2024-09-21T01:20:53.905 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:53 smithi027 ceph-mon[30375]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 6.4 KiB/s, 1 objects/s recovering 2024-09-21T01:20:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:53 smithi089 ceph-mon[34486]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 6.4 KiB/s, 1 objects/s recovering 2024-09-21T01:20:54.392 INFO:teuthology.orchestra.run.smithi027.stdout:111669149776 2024-09-21T01:20:54.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T01:20:54.757 INFO:teuthology.orchestra.run.smithi027.stdout:304942678052 2024-09-21T01:20:54.757 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T01:20:55.283 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.334 INFO:teuthology.orchestra.run.smithi027.stdout:257698037805 2024-09-21T01:20:55.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T01:20:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.471 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.479 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.517 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:55 smithi027 ceph-mon[30375]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s; 5.5 KiB/s, 1 objects/s recovering 2024-09-21T01:20:55.712 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:55 smithi098 ceph-mon[34372]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s; 5.5 KiB/s, 1 objects/s recovering 2024-09-21T01:20:55.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:55 smithi089 ceph-mon[34486]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s; 5.5 KiB/s, 1 objects/s recovering 2024-09-21T01:20:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 -- 172.21.15.27:0/3501459041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c105d80 msgr2=0x7fe75c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 --2- 172.21.15.27:0/3501459041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c105d80 0x7fe75c106190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009900 tx=0x7fe74c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 -- 172.21.15.27:0/3501459041 shutdown_connections 2024-09-21T01:20:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 --2- 172.21.15.27:0/3501459041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c10c010 0x7fe75c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 --2- 172.21.15.27:0/3501459041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10b6c0 0x7fe75c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 --2- 172.21.15.27:0/3501459041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c105d80 0x7fe75c106190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 -- 172.21.15.27:0/3501459041 >> 172.21.15.27:0/3501459041 conn(0x7fe75c101810 msgr2=0x7fe75c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 -- 172.21.15.27:0/3501459041 shutdown_connections 2024-09-21T01:20:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.413+0000 7fe760c00640 1 -- 172.21.15.27:0/3501459041 wait complete. 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.414+0000 7fe760c00640 1 Processor -- start 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 -- start start 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c105d80 0x7fe75c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c10b6c0 0x7fe75c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 0x7fe75c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c115cd0 con 0x7fe75c10b6c0 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c115b50 con 0x7fe75c10c010 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe760c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c115e50 con 0x7fe75c105d80 2024-09-21T01:20:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe75a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 0x7fe75c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe759e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c105d80 0x7fe75c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe753e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c10b6c0 0x7fe75c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.415+0000 7fe75a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 0x7fe75c1a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39912/0 (socket says 172.21.15.27:39912) 2024-09-21T01:20:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe75a800640 1 -- 172.21.15.27:0/3303430733 learned_addr learned my addr 172.21.15.27:0/3303430733 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe759e00640 1 -- 172.21.15.27:0/3303430733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 msgr2=0x7fe75c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe759e00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 0x7fe75c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe759e00640 1 -- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c10b6c0 msgr2=0x7fe75c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe759e00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c10b6c0 0x7fe75c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe759e00640 1 -- 172.21.15.27:0/3303430733 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c1a2690 con 0x7fe75c105d80 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe753e00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c10b6c0 0x7fe75c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe75a800640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 0x7fe75c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.416+0000 7fe759e00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c105d80 0x7fe75c19c720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe74c006010 tx=0x7fe74c004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.417+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c03f070 con 0x7fe75c105d80 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.417+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c1a2920 con 0x7fe75c105d80 2024-09-21T01:20:56.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.417+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c003ca0 con 0x7fe75c105d80 2024-09-21T01:20:56.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.417+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c035070 con 0x7fe75c105d80 2024-09-21T01:20:56.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.417+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a90e0 con 0x7fe75c105d80 2024-09-21T01:20:56.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.418+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c106190 con 0x7fe75c105d80 2024-09-21T01:20:56.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.421+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe74c03b460 con 0x7fe75c105d80 2024-09-21T01:20:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.422+0000 7fe752a00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe7400b65f0 0x7fe7400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.423+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe74c108100 con 0x7fe75c105d80 2024-09-21T01:20:56.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.423+0000 7fe753e00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe7400b65f0 0x7fe7400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.424+0000 7fe753e00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe7400b65f0 0x7fe7400b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7440095f0 tx=0x7fe744005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:56.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.424+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe74c0d2720 con 0x7fe75c105d80 2024-09-21T01:20:56.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 -- 172.21.15.27:0/1592710548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec281100d0 msgr2=0x7fec28112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 --2- 172.21.15.27:0/1592710548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec281100d0 0x7fec28112510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fec20009ca0 tx=0x7fec2002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 -- 172.21.15.27:0/1592710548 shutdown_connections 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 --2- 172.21.15.27:0/1592710548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec281100d0 0x7fec28112510 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 --2- 172.21.15.27:0/1592710548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec28076100 0x7fec2810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 --2- 172.21.15.27:0/1592710548 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec280757b0 0x7fec28075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 -- 172.21.15.27:0/1592710548 >> 172.21.15.27:0/1592710548 conn(0x7fec280ff5e0 msgr2=0x7fec28101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 -- 172.21.15.27:0/1592710548 shutdown_connections 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.297+0000 7fec2e200640 1 -- 172.21.15.27:0/1592710548 wait complete. 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 Processor -- start 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 -- start start 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec280757b0 0x7fec2819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28076100 0x7fec2819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec281100d0 0x7fec281a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec28115c80 con 0x7fec28076100 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fec28115b00 con 0x7fec281100d0 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.298+0000 7fec2e200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec28115e00 con 0x7fec280757b0 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28076100 0x7fec2819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28076100 0x7fec2819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53720/0 (socket says 172.21.15.27:53720) 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 -- 172.21.15.27:0/2149583297 learned_addr learned my addr 172.21.15.27:0/2149583297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec27400640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec280757b0 0x7fec2819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec27e00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec281100d0 0x7fec281a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 -- 172.21.15.27:0/2149583297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec280757b0 msgr2=0x7fec2819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec280757b0 0x7fec2819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 -- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec281100d0 msgr2=0x7fec281a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec281100d0 0x7fec281a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec26a00640 1 -- 172.21.15.27:0/2149583297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec281a2540 con 0x7fec28076100 2024-09-21T01:20:56.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.299+0000 7fec27400640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec280757b0 0x7fec2819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.300+0000 7fec26a00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28076100 0x7fec2819cb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e9c0 tx=0x7fec1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.300+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1400cd80 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.300+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec281a2830 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.300+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec140035f0 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.300+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14010690 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.300+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec281a8f90 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.301+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec28063b50 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.304+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fec14004000 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.305+0000 7fec1fe00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fec0c0b65f0 0x7fec0c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.305+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fec140e8190 con 0x7fec28076100 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.306+0000 7fec27400640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fec0c0b65f0 0x7fec0c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.306+0000 7fec1fe00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] conn(0x7fec0c0c1b00 0x7fec0c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.306+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 --> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec0c0c46c0 con 0x7fec0c0c1b00 2024-09-21T01:20:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.306+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fec140e85e0 con 0x7fec28076100 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.306+0000 7fec27e00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] conn(0x7fec0c0c1b00 0x7fec0c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.307+0000 7fec27400640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fec0c0b65f0 0x7fec0c0b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec180095f0 tx=0x7fec18005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.307+0000 7fec27e00640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] conn(0x7fec0c0c1b00 0x7fec0c0c3ff0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.308+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== osd.10 v2:172.21.15.89:6824/4138240961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec0c0c46c0 con 0x7fec0c0c1b00 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.323+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 --> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec28063fd0 con 0x7fec0c0c1b00 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.323+0000 7fec1fe00640 1 -- 172.21.15.27:0/2149583297 <== osd.10 v2:172.21.15.89:6824/4138240961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fec28063fd0 con 0x7fec0c0c1b00 2024-09-21T01:20:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] conn(0x7fec0c0c1b00 msgr2=0x7fec0c0c3ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] conn(0x7fec0c0c1b00 0x7fec0c0c3ff0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fec0c0b65f0 msgr2=0x7fec0c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fec0c0b65f0 0x7fec0c0b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec180095f0 tx=0x7fec18005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28076100 msgr2=0x7fec2819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 --2- 172.21.15.27:0/2149583297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec28076100 0x7fec2819cb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e9c0 tx=0x7fec1400ee80 comp rx=0 tx=0).stop 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec27e00640 1 -- 172.21.15.27:0/2149583297 reap_dead start 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 shutdown_connections 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 >> 172.21.15.27:0/2149583297 conn(0x7fec280ff5e0 msgr2=0x7fec28107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 shutdown_connections 2024-09-21T01:20:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.324+0000 7fec2e200640 1 -- 172.21.15.27:0/2149583297 wait complete. 2024-09-21T01:20:56.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.629+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe75c02e640 con 0x7fe75c105d80 2024-09-21T01:20:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.629+0000 7fe752a00640 1 -- 172.21.15.27:0/3303430733 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe74c0d7660 con 0x7fe75c105d80 2024-09-21T01:20:56.630 INFO:teuthology.orchestra.run.smithi027.stdout:137438953549 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe7400b65f0 msgr2=0x7fe7400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe7400b65f0 0x7fe7400b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7440095f0 tx=0x7fe744005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c105d80 msgr2=0x7fe75c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c105d80 0x7fe75c19c720 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe74c006010 tx=0x7fe74c004060 comp rx=0 tx=0).stop 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 shutdown_connections 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe7400b65f0 0x7fe7400b8b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe75c10c010 0x7fe75c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75c10b6c0 0x7fe75c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 --2- 172.21.15.27:0/3303430733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe75c105d80 0x7fe75c19c720 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 >> 172.21.15.27:0/3303430733 conn(0x7fe75c101810 msgr2=0x7fe75c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 shutdown_connections 2024-09-21T01:20:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:56.632+0000 7fe760c00640 1 -- 172.21.15.27:0/3303430733 wait complete. 2024-09-21T01:20:57.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:57 smithi027 ceph-mon[30375]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 5.5 KiB/s, 0 objects/s recovering 2024-09-21T01:20:57.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:57 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3303430733' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T01:20:57.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:57 smithi098 ceph-mon[34372]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 5.5 KiB/s, 0 objects/s recovering 2024-09-21T01:20:57.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:57 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3303430733' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T01:20:57.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:57 smithi089 ceph-mon[34486]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s; 5.5 KiB/s, 0 objects/s recovering 2024-09-21T01:20:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:57 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3303430733' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T01:20:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 -- 172.21.15.27:0/3324704316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 msgr2=0x7fd138110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 --2- 172.21.15.27:0/3324704316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 0x7fd138110470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd130009ca0 tx=0x7fd13002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 -- 172.21.15.27:0/3324704316 shutdown_connections 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 --2- 172.21.15.27:0/3324704316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 0x7fd138110470 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 --2- 172.21.15.27:0/3324704316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1381092f0 0x7fd138109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 --2- 172.21.15.27:0/3324704316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd138103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.600+0000 7fd13da00640 1 -- 172.21.15.27:0/3324704316 >> 172.21.15.27:0/3324704316 conn(0x7fd1380ff440 msgr2=0x7fd138101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.601+0000 7fd13da00640 1 -- 172.21.15.27:0/3324704316 shutdown_connections 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.601+0000 7fd13da00640 1 -- 172.21.15.27:0/3324704316 wait complete. 2024-09-21T01:20:58.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.602+0000 7fd13da00640 1 Processor -- start 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.602+0000 7fd13da00640 1 -- start start 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.602+0000 7fd13da00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd13819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd13da00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1381092f0 0x7fd13819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd13da00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 0x7fd1381a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd13da00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd138113900 con 0x7fd138109c40 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd13da00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd138113780 con 0x7fd1381039b0 2024-09-21T01:20:58.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd13da00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd138113a80 con 0x7fd1381092f0 2024-09-21T01:20:58.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd13819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd13d000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 0x7fd1381a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd13819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39938/0 (socket says 172.21.15.27:39938) 2024-09-21T01:20:58.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1381092f0 0x7fd13819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 -- 172.21.15.27:0/3875590398 learned_addr learned my addr 172.21.15.27:0/3875590398 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 -- 172.21.15.27:0/3875590398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1381092f0 msgr2=0x7fd13819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1381092f0 0x7fd13819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 -- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 msgr2=0x7fd1381a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.603+0000 7fd137e00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 0x7fd1381a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd137e00640 1 -- 172.21.15.27:0/3875590398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1381a23c0 con 0x7fd1381039b0 2024-09-21T01:20:58.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd137e00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd13819c450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd12800e9f0 tx=0x7fd12800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12800cd80 con 0x7fd1381039b0 2024-09-21T01:20:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1280108e0 con 0x7fd1381039b0 2024-09-21T01:20:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128018980 con 0x7fd1381039b0 2024-09-21T01:20:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1381a26b0 con 0x7fd1381039b0 2024-09-21T01:20:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.604+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1381a8e10 con 0x7fd1381039b0 2024-09-21T01:20:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.606+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd138103dc0 con 0x7fd1381039b0 2024-09-21T01:20:58.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.610+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fd128010430 con 0x7fd1381039b0 2024-09-21T01:20:58.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.612+0000 7fd134c00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd11c0b65f0 0x7fd11c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.612+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd1280e7d20 con 0x7fd1381039b0 2024-09-21T01:20:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.612+0000 7fd137400640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd11c0b65f0 0x7fd11c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:20:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.613+0000 7fd137400640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd11c0b65f0 0x7fd11c0b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd138108ae0 tx=0x7fd124005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:58.616 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.615+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1280b2340 con 0x7fd1381039b0 2024-09-21T01:20:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.832+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd1381a9180 con 0x7fd1381039b0 2024-09-21T01:20:58.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.833+0000 7fd134c00640 1 -- 172.21.15.27:0/3875590398 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1280b7280 con 0x7fd1381039b0 2024-09-21T01:20:58.834 INFO:teuthology.orchestra.run.smithi027.stdout:158913790024 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd11c0b65f0 msgr2=0x7fd11c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd11c0b65f0 0x7fd11c0b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd138108ae0 tx=0x7fd124005f90 comp rx=0 tx=0).stop 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 msgr2=0x7fd13819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd13819c450 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd12800e9f0 tx=0x7fd12800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 shutdown_connections 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd11c0b65f0 0x7fd11c0b8b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd138109c40 0x7fd1381a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1381092f0 0x7fd13819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 --2- 172.21.15.27:0/3875590398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1381039b0 0x7fd13819c450 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 >> 172.21.15.27:0/3875590398 conn(0x7fd1380ff440 msgr2=0x7fd138106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 shutdown_connections 2024-09-21T01:20:58.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.836+0000 7fd13da00640 1 -- 172.21.15.27:0/3875590398 wait complete. 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 -- 172.21.15.27:0/360078416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941092f0 msgr2=0x7f8694109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 --2- 172.21.15.27:0/360078416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941092f0 0x7f8694109700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f867c009900 tx=0x7f867c02a410 comp rx=0 tx=0).stop 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 -- 172.21.15.27:0/360078416 shutdown_connections 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 --2- 172.21.15.27:0/360078416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8694109c40 0x7f8694110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 --2- 172.21.15.27:0/360078416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941092f0 0x7f8694109700 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 --2- 172.21.15.27:0/360078416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86941039b0 0x7f8694103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 -- 172.21.15.27:0/360078416 >> 172.21.15.27:0/360078416 conn(0x7f86940ff440 msgr2=0x7f8694101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.496+0000 7f8693200640 1 -- 172.21.15.27:0/360078416 shutdown_connections 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.497+0000 7f8693200640 1 -- 172.21.15.27:0/360078416 wait complete. 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 Processor -- start 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 -- start start 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 0x7f869419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f86941092f0 0x7f869419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8694109c40 0x7f86941a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8694113950 con 0x7f8694109c40 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f86941137d0 con 0x7f86941092f0 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.498+0000 7f8693200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8694113ad0 con 0x7f86941039b0 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 0x7f869419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 0x7f869419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56564/0 (socket says 172.21.15.27:56564) 2024-09-21T01:20:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 -- 172.21.15.27:0/1582071206 learned_addr learned my addr 172.21.15.27:0/1582071206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8692800640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8694109c40 0x7f86941a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8689400640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f86941092f0 0x7f869419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 -- 172.21.15.27:0/1582071206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f86941092f0 msgr2=0x7f869419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f86941092f0 0x7f869419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 -- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8694109c40 msgr2=0x7f86941a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8694109c40 0x7f86941a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.499+0000 7f8691e00640 1 -- 172.21.15.27:0/1582071206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86941a2470 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.500+0000 7f8691e00640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 0x7f869419c500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.500+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400cd80 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.500+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86840108e0 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.500+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684018980 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.501+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86941a2760 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.501+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941a8ec0 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.502+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8694104c30 con 0x7f86941039b0 2024-09-21T01:20:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.504+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8684010430 con 0x7f86941039b0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.506+0000 7f868b400640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f86780b6670 0x7f86780b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.506+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f86840e8120 con 0x7f86941039b0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.506+0000 7f8689400640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f86780b6670 0x7f86780b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.507+0000 7f8689400640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f86780b6670 0x7f86780b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f867c02aa00 tx=0x7f867c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.507+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86840b2740 con 0x7f86941039b0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.741+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f86941a9120 con 0x7f86941039b0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.742+0000 7f868b400640 1 -- 172.21.15.27:0/1582071206 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86840b7680 con 0x7f86941039b0 2024-09-21T01:20:58.990 INFO:teuthology.orchestra.run.smithi027.stdout:60129542237 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f86780b6670 msgr2=0x7f86780b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f86780b6670 0x7f86780b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f867c02aa00 tx=0x7f867c005a20 comp rx=0 tx=0).stop 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 msgr2=0x7f869419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 0x7f869419c500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 shutdown_connections 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f86780b6670 0x7f86780b8b80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8694109c40 0x7f86941a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f86941092f0 0x7f869419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 --2- 172.21.15.27:0/1582071206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86941039b0 0x7f869419c500 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 >> 172.21.15.27:0/1582071206 conn(0x7f86940ff440 msgr2=0x7f8694108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 shutdown_connections 2024-09-21T01:20:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:58.744+0000 7f8693200640 1 -- 172.21.15.27:0/1582071206 wait complete. 2024-09-21T01:20:59.601 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:59 smithi027 ceph-mon[30375]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T01:20:59.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:59 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1582071206' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T01:20:59.602 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:20:59 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3875590398' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T01:20:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:59 smithi098 ceph-mon[34372]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T01:20:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:59 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1582071206' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T01:20:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:20:59 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3875590398' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T01:20:59.967 INFO:teuthology.orchestra.run.smithi027.stdout:279172874282 2024-09-21T01:20:59.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T01:21:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:59 smithi089 ceph-mon[34486]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T01:21:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:59 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1582071206' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T01:21:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:20:59 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3875590398' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 -- 172.21.15.27:0/3545331451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8102460 msgr2=0x7f91c8102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 --2- 172.21.15.27:0/3545331451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8102460 0x7f91c8102870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009900 tx=0x7f91bc02a480 comp rx=0 tx=0).stop 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 -- 172.21.15.27:0/3545331451 shutdown_connections 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 --2- 172.21.15.27:0/3545331451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8110010 0x7f91c8112430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 --2- 172.21.15.27:0/3545331451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102db0 0x7f91c810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 --2- 172.21.15.27:0/3545331451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8102460 0x7f91c8102870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 -- 172.21.15.27:0/3545331451 >> 172.21.15.27:0/3545331451 conn(0x7f91c80fe2c0 msgr2=0x7f91c8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 -- 172.21.15.27:0/3545331451 shutdown_connections 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.526+0000 7f91cf200640 1 -- 172.21.15.27:0/3545331451 wait complete. 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.527+0000 7f91cf200640 1 Processor -- start 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.527+0000 7f91cf200640 1 -- start start 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cf200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 0x7f91c819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cf200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8102db0 0x7f91c819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cf200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8110010 0x7f91c81a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cf200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91c8115ad0 con 0x7f91c8102db0 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cf200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f91c8115950 con 0x7f91c8110010 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cf200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91c8115c50 con 0x7f91c8102460 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 0x7f91c819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cd400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8102db0 0x7f91c819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cde00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 0x7f91c819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56584/0 (socket says 172.21.15.27:56584) 2024-09-21T01:21:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91cde00640 1 -- 172.21.15.27:0/2867903407 learned_addr learned my addr 172.21.15.27:0/2867903407 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.528+0000 7f91ce800640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8110010 0x7f91c81a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cd400640 1 -- 172.21.15.27:0/2867903407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 msgr2=0x7f91c819c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cd400640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 0x7f91c819c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cd400640 1 -- 172.21.15.27:0/2867903407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8110010 msgr2=0x7f91c81a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cd400640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8110010 0x7f91c81a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cd400640 1 -- 172.21.15.27:0/2867903407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c81a23b0 con 0x7f91c8102db0 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cde00640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 0x7f91c819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cd400640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8102db0 0x7f91c819c9e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91b800e9f0 tx=0x7f91b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b800ccc0 con 0x7f91c8102db0 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c81a26a0 con 0x7f91c8102db0 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b80108e0 con 0x7f91c8102db0 2024-09-21T01:21:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8018880 con 0x7f91c8102db0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stdout:111669149778 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.529+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c81a8e00 con 0x7f91c8102db0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.530+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c8102870 con 0x7f91c8102db0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.533+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f91b8010430 con 0x7f91c8102db0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.534+0000 7f91c2a00640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f91b00b6670 0x7f91b00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.535+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f91b80e81c0 con 0x7f91c8102db0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.535+0000 7f91cde00640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f91b00b6670 0x7f91b00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.536+0000 7f91cde00640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f91b00b6670 0x7f91b00b8b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0096c0 tx=0x7f91bc005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.537+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91b80b2840 con 0x7f91c8102db0 2024-09-21T01:21:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.742+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f91c810ac10 con 0x7f91c8102db0 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.743+0000 7f91c2a00640 1 -- 172.21.15.27:0/2867903407 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f91b80b7780 con 0x7f91c8102db0 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f91b00b6670 msgr2=0x7f91b00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f91b00b6670 0x7f91b00b8b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0096c0 tx=0x7f91bc005a20 comp rx=0 tx=0).stop 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8102db0 msgr2=0x7f91c819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8102db0 0x7f91c819c9e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f91b800e9f0 tx=0x7f91b800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 shutdown_connections 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f91b00b6670 0x7f91b00b8b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f91c8110010 0x7f91c81a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91c8102db0 0x7f91c819c9e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 --2- 172.21.15.27:0/2867903407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91c8102460 0x7f91c819c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 >> 172.21.15.27:0/2867903407 conn(0x7f91c80fe2c0 msgr2=0x7f91c81120d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 shutdown_connections 2024-09-21T01:21:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:20:59.745+0000 7f91cf200640 1 -- 172.21.15.27:0/2867903407 wait complete. 2024-09-21T01:21:00.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:00 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2867903407' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 -- 172.21.15.27:0/2114604200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99301094a0 msgr2=0x7f99301098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 --2- 172.21.15.27:0/2114604200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99301094a0 0x7f99301098b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9918009900 tx=0x7f991802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 -- 172.21.15.27:0/2114604200 shutdown_connections 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 --2- 172.21.15.27:0/2114604200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9930109df0 0x7f9930110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 --2- 172.21.15.27:0/2114604200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99301094a0 0x7f99301098b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 --2- 172.21.15.27:0/2114604200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9930103b60 0x7f9930103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.560+0000 7f9936a00640 1 -- 172.21.15.27:0/2114604200 >> 172.21.15.27:0/2114604200 conn(0x7f993007b850 msgr2=0x7f993007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:00.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.561+0000 7f9936a00640 1 -- 172.21.15.27:0/2114604200 shutdown_connections 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.561+0000 7f9936a00640 1 -- 172.21.15.27:0/2114604200 wait complete. 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 Processor -- start 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 -- start start 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9930103b60 0x7f993019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 0x7f993019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9930109df0 0x7f99301a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9930113ab0 con 0x7f9930109df0 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9930113930 con 0x7f99301094a0 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9936a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9930113c30 con 0x7f9930103b60 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9934a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9930109df0 0x7f99301a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.562+0000 7f9927400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 0x7f993019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 0x7f993019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39976/0 (socket says 172.21.15.27:39976) 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 -- 172.21.15.27:0/3268235295 learned_addr learned my addr 172.21.15.27:0/3268235295 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f992fe00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9930103b60 0x7f993019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 -- 172.21.15.27:0/3268235295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9930103b60 msgr2=0x7f993019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9930103b60 0x7f993019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 -- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9930109df0 msgr2=0x7f99301a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9930109df0 0x7f99301a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 -- 172.21.15.27:0/3268235295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99301a2650 con 0x7f99301094a0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.563+0000 7f9927400640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 0x7f993019cc80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f991802a910 tx=0x7f9918031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.564+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991803f030 con 0x7f99301094a0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.564+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99180043d0 con 0x7f99301094a0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.564+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99301a28e0 con 0x7f99301094a0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.564+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918035070 con 0x7f99301094a0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.564+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99301a90a0 con 0x7f99301094a0 2024-09-21T01:21:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.565+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9930103f70 con 0x7f99301094a0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stdout:184683593794 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.567+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9918042030 con 0x7f99301094a0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.569+0000 7f992d600640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99140b6670 0x7f99140b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.569+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9918107c00 con 0x7f99301094a0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.569+0000 7f992fe00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99140b6670 0x7f99140b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.570+0000 7f992fe00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99140b6670 0x7f99140b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9920009bd0 tx=0x7f9920005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.571+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99180cde40 con 0x7f99301094a0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.773+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f99301a92a0 con 0x7f99301094a0 2024-09-21T01:21:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.774+0000 7f992d600640 1 -- 172.21.15.27:0/3268235295 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99180d2220 con 0x7f99301094a0 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99140b6670 msgr2=0x7f99140b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99140b6670 0x7f99140b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9920009bd0 tx=0x7f9920005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 msgr2=0x7f993019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 0x7f993019cc80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f991802a910 tx=0x7f9918031a20 comp rx=0 tx=0).stop 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 shutdown_connections 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99140b6670 0x7f99140b8b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9930109df0 0x7f99301a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99301094a0 0x7f993019cc80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 --2- 172.21.15.27:0/3268235295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9930103b60 0x7f993019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 >> 172.21.15.27:0/3268235295 conn(0x7f993007b850 msgr2=0x7f9930105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 shutdown_connections 2024-09-21T01:21:00.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:00.776+0000 7f9936a00640 1 -- 172.21.15.27:0/3268235295 wait complete. 2024-09-21T01:21:00.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:00 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2867903407' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T01:21:01.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:00 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2867903407' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T01:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 -- 172.21.15.27:0/1961993201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 msgr2=0x7fde40105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 --2- 172.21.15.27:0/1961993201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde40105fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde34009860 tx=0x7fde3402a340 comp rx=0 tx=0).stop 2024-09-21T01:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 -- 172.21.15.27:0/1961993201 shutdown_connections 2024-09-21T01:21:01.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 --2- 172.21.15.27:0/1961993201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4010be40 0x7fde40112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 --2- 172.21.15.27:0/1961993201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 --2- 172.21.15.27:0/1961993201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde40105fc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 -- 172.21.15.27:0/1961993201 >> 172.21.15.27:0/1961993201 conn(0x7fde40101680 msgr2=0x7fde40103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:01.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 -- 172.21.15.27:0/1961993201 shutdown_connections 2024-09-21T01:21:01.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.364+0000 7fde46000640 1 -- 172.21.15.27:0/1961993201 wait complete. 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.365+0000 7fde46000640 1 Processor -- start 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 -- start start 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde4019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4010be40 0x7fde401a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde40115b00 con 0x7fde4010be40 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde40115980 con 0x7fde40105bb0 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde46000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde40115c80 con 0x7fde4010b4f0 2024-09-21T01:21:01.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde45600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4010be40 0x7fde401a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56620/0 (socket says 172.21.15.27:56620) 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.366+0000 7fde3fe00640 1 -- 172.21.15.27:0/1115088663 learned_addr learned my addr 172.21.15.27:0/1115088663 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde44c00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde4019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde3fe00640 1 -- 172.21.15.27:0/1115088663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 msgr2=0x7fde4019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde3fe00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde4019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde3fe00640 1 -- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4010be40 msgr2=0x7fde401a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde3fe00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4010be40 0x7fde401a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde3fe00640 1 -- 172.21.15.27:0/1115088663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde401a24c0 con 0x7fde4010b4f0 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde44c00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde4019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:01.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.367+0000 7fde3fe00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4019caf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c3a0 tx=0x7fde2c00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.368+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c005020 con 0x7fde4010b4f0 2024-09-21T01:21:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.368+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde401a27b0 con 0x7fde4010b4f0 2024-09-21T01:21:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.368+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2c005670 con 0x7fde4010b4f0 2024-09-21T01:21:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.368+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2c01e740 con 0x7fde4010b4f0 2024-09-21T01:21:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.368+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde40077800 con 0x7fde4010b4f0 2024-09-21T01:21:01.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.369+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde40106e30 con 0x7fde4010b4f0 2024-09-21T01:21:01.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.372+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fde2c0046c0 con 0x7fde4010b4f0 2024-09-21T01:21:01.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.373+0000 7fde3d600640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde280b65f0 0x7fde280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.373+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fde2c0f0520 con 0x7fde4010b4f0 2024-09-21T01:21:01.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.374+0000 7fde44c00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde280b65f0 0x7fde280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.374+0000 7fde44c00640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde280b65f0 0x7fde280b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde3402a930 tx=0x7fde340058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:01.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.375+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde2c0bab40 con 0x7fde4010b4f0 2024-09-21T01:21:01.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.585+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fde4010dc10 con 0x7fde4010b4f0 2024-09-21T01:21:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.585+0000 7fde3d600640 1 -- 172.21.15.27:0/1115088663 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fde2c0bfa80 con 0x7fde4010b4f0 2024-09-21T01:21:01.586 INFO:teuthology.orchestra.run.smithi027.stdout:231928234037 2024-09-21T01:21:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde280b65f0 msgr2=0x7fde280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde280b65f0 0x7fde280b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fde3402a930 tx=0x7fde340058d0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 msgr2=0x7fde4019caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4019caf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fde2c00c3a0 tx=0x7fde2c00c860 comp rx=0 tx=0).stop 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 shutdown_connections 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde280b65f0 0x7fde280b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4010be40 0x7fde401a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4010b4f0 0x7fde4019caf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 --2- 172.21.15.27:0/1115088663 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde40105bb0 0x7fde4019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 >> 172.21.15.27:0/1115088663 conn(0x7fde40101680 msgr2=0x7fde40103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 shutdown_connections 2024-09-21T01:21:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.588+0000 7fde46000640 1 -- 172.21.15.27:0/1115088663 wait complete. 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 -- 172.21.15.27:0/3481016714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010b6f0 msgr2=0x7fa4e010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3481016714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010b6f0 0x7fa4e010bb00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 -- 172.21.15.27:0/3481016714 shutdown_connections 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3481016714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010c040 0x7fa4e0112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3481016714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010b6f0 0x7fa4e010bb00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3481016714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 0x7fa4e01061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 -- 172.21.15.27:0/3481016714 >> 172.21.15.27:0/3481016714 conn(0x7fa4e01017e0 msgr2=0x7fa4e0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 -- 172.21.15.27:0/3481016714 shutdown_connections 2024-09-21T01:21:01.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.759+0000 7fa4e5600640 1 -- 172.21.15.27:0/3481016714 wait complete. 2024-09-21T01:21:01.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.760+0000 7fa4e5600640 1 Processor -- start 2024-09-21T01:21:01.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.760+0000 7fa4e5600640 1 -- start start 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4e5600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 0x7fa4e019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4e5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 0x7fa4e019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4e5600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010c040 0x7fa4e01a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4e5600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0115d50 con 0x7fa4e010b6f0 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4e5600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0115bd0 con 0x7fa4e0105db0 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4e5600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e0115ed0 con 0x7fa4e010c040 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 0x7fa4e019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 0x7fa4e019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53836/0 (socket says 172.21.15.27:53836) 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 -- 172.21.15.27:0/3561774887 learned_addr learned my addr 172.21.15.27:0/3561774887 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4df400640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010c040 0x7fa4e01a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4dea00640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 0x7fa4e019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 -- 172.21.15.27:0/3561774887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010c040 msgr2=0x7fa4e01a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010c040 0x7fa4e01a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 -- 172.21.15.27:0/3561774887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 msgr2=0x7fa4e019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.761+0000 7fa4de000640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 0x7fa4e019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4de000640 1 -- 172.21.15.27:0/3561774887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e01a26f0 con 0x7fa4e010b6f0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4df400640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010c040 0x7fa4e01a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4dea00640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 0x7fa4e019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4de000640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 0x7fa4e019cd20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc005ab0 tx=0x7fa4cc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc03f070 con 0x7fa4e010b6f0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4cc031b50 con 0x7fa4e010b6f0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc035030 con 0x7fa4e010b6f0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e01a2980 con 0x7fa4e010b6f0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.762+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e01a9140 con 0x7fa4e010b6f0 2024-09-21T01:21:01.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.764+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e01061c0 con 0x7fa4e010b6f0 2024-09-21T01:21:01.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.766+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fa4cc042070 con 0x7fa4e010b6f0 2024-09-21T01:21:01.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.767+0000 7fa4d7400640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa4c40b65f0 0x7fa4c40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:01.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.768+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa4cc108380 con 0x7fa4e010b6f0 2024-09-21T01:21:01.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.768+0000 7fa4dea00640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa4c40b65f0 0x7fa4c40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:01.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.769+0000 7fa4dea00640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa4c40b65f0 0x7fa4c40b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009fc0 tx=0x7fa4d0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:01.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.770+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa4cc0ce5c0 con 0x7fa4e010b6f0 2024-09-21T01:21:01.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:01 smithi098 ceph-mon[34372]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T01:21:01.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:01 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3268235295' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T01:21:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:01 smithi027 ceph-mon[30375]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T01:21:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:01 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3268235295' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T01:21:01.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.974+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa4e002e640 con 0x7fa4e010b6f0 2024-09-21T01:21:01.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.975+0000 7fa4d7400640 1 -- 172.21.15.27:0/3561774887 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4cc0d29a0 con 0x7fa4e010b6f0 2024-09-21T01:21:01.975 INFO:teuthology.orchestra.run.smithi027.stdout:81604378712 2024-09-21T01:21:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa4c40b65f0 msgr2=0x7fa4c40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa4c40b65f0 0x7fa4c40b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009fc0 tx=0x7fa4d0005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 msgr2=0x7fa4e019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 0x7fa4e019cd20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc005ab0 tx=0x7fa4cc003b90 comp rx=0 tx=0).stop 2024-09-21T01:21:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 shutdown_connections 2024-09-21T01:21:01.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa4c40b65f0 0x7fa4c40b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4e010c040 0x7fa4e01a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4e010b6f0 0x7fa4e019cd20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 --2- 172.21.15.27:0/3561774887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa4e0105db0 0x7fa4e019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:01.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.977+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 >> 172.21.15.27:0/3561774887 conn(0x7fa4e01017e0 msgr2=0x7fa4e0108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:01.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.978+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 shutdown_connections 2024-09-21T01:21:01.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:01.978+0000 7fa4e5600640 1 -- 172.21.15.27:0/3561774887 wait complete. 2024-09-21T01:21:01.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:01 smithi089 ceph-mon[34486]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-21T01:21:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:01 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3268235295' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T01:21:02.651 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:02 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1115088663' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T01:21:02.651 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:02 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3561774887' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T01:21:02.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:02 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1115088663' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T01:21:02.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:02 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3561774887' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T01:21:02.901 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:02 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1115088663' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T01:21:02.901 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:02 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3561774887' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T01:21:03.481 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234034 got 231928234037 for osd.8 2024-09-21T01:21:03.481 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:03.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378709 got 81604378712 for osd.2 2024-09-21T01:21:03.626 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:03.743 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:03 smithi027 ceph-mon[30375]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:03.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953546 got 137438953549 for osd.4 2024-09-21T01:21:03.786 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:03.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:03 smithi098 ceph-mon[34372]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:04.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:03 smithi089 ceph-mon[34486]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:04.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 -- 172.21.15.27:0/3131153338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 msgr2=0x7f33f8112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 --2- 172.21.15.27:0/3131153338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 0x7f33f8112350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009ca0 tx=0x7f33f002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 -- 172.21.15.27:0/3131153338 shutdown_connections 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 --2- 172.21.15.27:0/3131153338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 0x7f33f8112350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 --2- 172.21.15.27:0/3131153338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33f8076100 0x7f33f810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 --2- 172.21.15.27:0/3131153338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 -- 172.21.15.27:0/3131153338 >> 172.21.15.27:0/3131153338 conn(0x7f33f80ff440 msgr2=0x7f33f8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 -- 172.21.15.27:0/3131153338 shutdown_connections 2024-09-21T01:21:04.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.268+0000 7f33fe000640 1 -- 172.21.15.27:0/3131153338 wait complete. 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.269+0000 7f33fe000640 1 Processor -- start 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 -- start start 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33f8076100 0x7f33f819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 0x7f33f81a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8115ac0 con 0x7f33f810ff10 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8115940 con 0x7f33f8076100 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fe000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33f8115c40 con 0x7f33f80757b0 2024-09-21T01:21:04.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fcc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fcc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56674/0 (socket says 172.21.15.27:56674) 2024-09-21T01:21:04.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fcc00640 1 -- 172.21.15.27:0/3772487100 learned_addr learned my addr 172.21.15.27:0/3772487100 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:04.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33fd600640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 0x7f33f81a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.270+0000 7f33f7e00640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33f8076100 0x7f33f819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fcc00640 1 -- 172.21.15.27:0/3772487100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33f8076100 msgr2=0x7f33f819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fcc00640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33f8076100 0x7f33f819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fcc00640 1 -- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 msgr2=0x7f33f81a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fcc00640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 0x7f33f81a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fcc00640 1 -- 172.21.15.27:0/3772487100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f81a2460 con 0x7f33f80757b0 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fcc00640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f819c4f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00e9f0 tx=0x7f33ec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00cd80 con 0x7f33f80757b0 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f81a2750 con 0x7f33f80757b0 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.271+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33ec0054d0 con 0x7f33f80757b0 2024-09-21T01:21:04.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.272+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec010690 con 0x7f33f80757b0 2024-09-21T01:21:04.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.272+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f81a8eb0 con 0x7f33f80757b0 2024-09-21T01:21:04.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.273+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f8075bc0 con 0x7f33f80757b0 2024-09-21T01:21:04.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.276+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f33ec004e90 con 0x7f33f80757b0 2024-09-21T01:21:04.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.277+0000 7f33f5600640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f33e00b65f0 0x7f33e00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.277+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f33ec0e80c0 con 0x7f33f80757b0 2024-09-21T01:21:04.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.277+0000 7f33f7e00640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f33e00b65f0 0x7f33e00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.278+0000 7f33f7e00640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f33e00b65f0 0x7f33e00b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33e40095f0 tx=0x7f33e4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:04.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.279+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33ec0ae300 con 0x7f33f80757b0 2024-09-21T01:21:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 -- 172.21.15.27:0/1887249859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e4105d90 msgr2=0x7fc4e41061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 --2- 172.21.15.27:0/1887249859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e4105d90 0x7fc4e41061a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009900 tx=0x7fc4d802a410 comp rx=0 tx=0).stop 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 -- 172.21.15.27:0/1887249859 shutdown_connections 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 --2- 172.21.15.27:0/1887249859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 0x7fc4e4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 --2- 172.21.15.27:0/1887249859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e410b6d0 0x7fc4e410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 --2- 172.21.15.27:0/1887249859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e4105d90 0x7fc4e41061a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 -- 172.21.15.27:0/1887249859 >> 172.21.15.27:0/1887249859 conn(0x7fc4e4101860 msgr2=0x7fc4e4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 -- 172.21.15.27:0/1887249859 shutdown_connections 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.164+0000 7fc4ec000640 1 -- 172.21.15.27:0/1887249859 wait complete. 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.165+0000 7fc4ec000640 1 Processor -- start 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.165+0000 7fc4ec000640 1 -- start start 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ec000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e4105d90 0x7fc4e419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ec000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e410b6d0 0x7fc4e419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ec000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 0x7fc4e41a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ec000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4115d30 con 0x7fc4e410b6d0 2024-09-21T01:21:04.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ec000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4115bb0 con 0x7fc4e4105d90 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ec000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4115eb0 con 0x7fc4e410c020 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ea000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 0x7fc4e41a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4e8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e410b6d0 0x7fc4e419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ea000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 0x7fc4e41a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:56652/0 (socket says 172.21.15.27:56652) 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4e8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e410b6d0 0x7fc4e419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53846/0 (socket says 172.21.15.27:53846) 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4ea000640 1 -- 172.21.15.27:0/2526222903 learned_addr learned my addr 172.21.15.27:0/2526222903 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.166+0000 7fc4e9600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e4105d90 0x7fc4e419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e9600640 1 -- 172.21.15.27:0/2526222903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 msgr2=0x7fc4e41a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e9600640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 0x7fc4e41a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e9600640 1 -- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e410b6d0 msgr2=0x7fc4e419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e9600640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e410b6d0 0x7fc4e419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e9600640 1 -- 172.21.15.27:0/2526222903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e41a2730 con 0x7fc4e4105d90 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e9600640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e4105d90 0x7fc4e419c7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4d802aa00 tx=0x7fc4d8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:04.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d803f070 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4d8031b50 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e41a29c0 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8035030 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.167+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41a9180 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.169+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e41061a0 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.171+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc4d8042070 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.173+0000 7fc4e2000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4cc0b65f0 0x7fc4cc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.173+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fc4d8107c70 con 0x7fc4e4105d90 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.173+0000 7fc4e8c00640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4cc0b65f0 0x7fc4cc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.174+0000 7fc4e8c00640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4cc0b65f0 0x7fc4cc0b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4104f60 tx=0x7fc4d0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:04.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.178+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc4d8031d00 con 0x7fc4e4105d90 2024-09-21T01:21:04.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.387+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc4e410de20 con 0x7fc4e4105d90 2024-09-21T01:21:04.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.388+0000 7fc4e2000640 1 -- 172.21.15.27:0/2526222903 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4d80cdeb0 con 0x7fc4e4105d90 2024-09-21T01:21:04.388 INFO:teuthology.orchestra.run.smithi027.stdout:38654705764 2024-09-21T01:21:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4cc0b65f0 msgr2=0x7fc4cc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4cc0b65f0 0x7fc4cc0b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4104f60 tx=0x7fc4d0005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e4105d90 msgr2=0x7fc4e419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e4105d90 0x7fc4e419c7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc4d802aa00 tx=0x7fc4d8003b90 comp rx=0 tx=0).stop 2024-09-21T01:21:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 shutdown_connections 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc4cc0b65f0 0x7fc4cc0b8b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4e410c020 0x7fc4e41a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.390+0000 7fc4ec000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e410b6d0 0x7fc4e419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.391+0000 7fc4ec000640 1 --2- 172.21.15.27:0/2526222903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc4e4105d90 0x7fc4e419c7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.391+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 >> 172.21.15.27:0/2526222903 conn(0x7fc4e4101860 msgr2=0x7fc4e410ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.391+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 shutdown_connections 2024-09-21T01:21:04.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.391+0000 7fc4ec000640 1 -- 172.21.15.27:0/2526222903 wait complete. 2024-09-21T01:21:04.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.487+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f33f819d620 con 0x7f33f80757b0 2024-09-21T01:21:04.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.488+0000 7f33f5600640 1 -- 172.21.15.27:0/3772487100 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f33f819d620 con 0x7f33f80757b0 2024-09-21T01:21:04.489 INFO:teuthology.orchestra.run.smithi027.stdout:206158430268 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.491+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f33e00b65f0 msgr2=0x7f33e00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.491+0000 7f33fe000640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f33e00b65f0 0x7f33e00b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33e40095f0 tx=0x7f33e4005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.491+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 msgr2=0x7f33f819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.491+0000 7f33fe000640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f819c4f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00e9f0 tx=0x7f33ec00edf0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 shutdown_connections 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f33e00b65f0 0x7f33e00b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33f810ff10 0x7f33f81a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f33f8076100 0x7f33f819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 --2- 172.21.15.27:0/3772487100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33f80757b0 0x7f33f819c4f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 >> 172.21.15.27:0/3772487100 conn(0x7f33f80ff440 msgr2=0x7f33f8109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 shutdown_connections 2024-09-21T01:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:04.492+0000 7f33fe000640 1 -- 172.21.15.27:0/3772487100 wait complete. 2024-09-21T01:21:04.633 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:04 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2526222903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T01:21:04.634 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:04 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3772487100' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T01:21:04.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:04 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2526222903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T01:21:04.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:04 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3772487100' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T01:21:05.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:04 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2526222903' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T01:21:05.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:04 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3772487100' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T01:21:05.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.182+0000 7ff02ea00640 1 -- 172.21.15.27:0/1285430765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0280757b0 msgr2=0x7ff028075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:05.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.182+0000 7ff02ea00640 1 --2- 172.21.15.27:0/1285430765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0280757b0 0x7ff028075bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff018009900 tx=0x7ff01802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 -- 172.21.15.27:0/1285430765 shutdown_connections 2024-09-21T01:21:05.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 --2- 172.21.15.27:0/1285430765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 0x7ff028112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 --2- 172.21.15.27:0/1285430765 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff028076100 0x7ff02810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 --2- 172.21.15.27:0/1285430765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0280757b0 0x7ff028075bc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 -- 172.21.15.27:0/1285430765 >> 172.21.15.27:0/1285430765 conn(0x7ff0280ff5e0 msgr2=0x7ff028101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 -- 172.21.15.27:0/1285430765 shutdown_connections 2024-09-21T01:21:05.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.183+0000 7ff02ea00640 1 -- 172.21.15.27:0/1285430765 wait complete. 2024-09-21T01:21:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.185+0000 7ff02ea00640 1 Processor -- start 2024-09-21T01:21:05.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.185+0000 7ff02ea00640 1 -- start start 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.185+0000 7ff02ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0280757b0 0x7ff02819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.185+0000 7ff02ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 0x7ff02819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.185+0000 7ff02ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 0x7ff0281a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff02ea00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff028115cd0 con 0x7ff028076100 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff02ea00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff028115b50 con 0x7ff0280757b0 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff02ea00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff028115e50 con 0x7ff0281100d0 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff027400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 0x7ff02819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff02ca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 0x7ff0281a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff027400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 0x7ff02819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53898/0 (socket says 172.21.15.27:53898) 2024-09-21T01:21:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff027400640 1 -- 172.21.15.27:0/2175729951 learned_addr learned my addr 172.21.15.27:0/2175729951 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.186+0000 7ff027e00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0280757b0 0x7ff02819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff02ca00640 1 -- 172.21.15.27:0/2175729951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0280757b0 msgr2=0x7ff02819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff02ca00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0280757b0 0x7ff02819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff02ca00640 1 -- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 msgr2=0x7ff02819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff02ca00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 0x7ff02819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff02ca00640 1 -- 172.21.15.27:0/2175729951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0281a25b0 con 0x7ff0281100d0 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff027400640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 0x7ff02819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff02ca00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 0x7ff0281a1e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff020009fd0 tx=0x7ff02000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.187+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02000ce20 con 0x7ff0281100d0 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.188+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0281a28a0 con 0x7ff0281100d0 2024-09-21T01:21:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.188+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff020005020 con 0x7ff0281100d0 2024-09-21T01:21:05.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.188+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020010690 con 0x7ff0281100d0 2024-09-21T01:21:05.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.188+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281a9000 con 0x7ff0281100d0 2024-09-21T01:21:05.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.189+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff028075bc0 con 0x7ff0281100d0 2024-09-21T01:21:05.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.192+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff0200054d0 con 0x7ff0281100d0 2024-09-21T01:21:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.194+0000 7ff024c00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff00c0b6670 0x7ff00c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.194+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff0200e8070 con 0x7ff0281100d0 2024-09-21T01:21:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.195+0000 7ff027e00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff00c0b6670 0x7ff00c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.196+0000 7ff027e00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff00c0b6670 0x7ff00c0b8b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff018005ab0 tx=0x7ff018005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.197+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0200ae2b0 con 0x7ff0281100d0 2024-09-21T01:21:05.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.414+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff02819d770 con 0x7ff0281100d0 2024-09-21T01:21:05.415 INFO:teuthology.orchestra.run.smithi027.stdout:304942678055 2024-09-21T01:21:05.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.414+0000 7ff024c00640 1 -- 172.21.15.27:0/2175729951 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff0200b2690 con 0x7ff0281100d0 2024-09-21T01:21:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff00c0b6670 msgr2=0x7ff00c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:05.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff00c0b6670 0x7ff00c0b8b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff018005ab0 tx=0x7ff018005a20 comp rx=0 tx=0).stop 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 msgr2=0x7ff0281a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 0x7ff0281a1e80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff020009fd0 tx=0x7ff02000ef10 comp rx=0 tx=0).stop 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 shutdown_connections 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff00c0b6670 0x7ff00c0b8b80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0281100d0 0x7ff0281a1e80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff028076100 0x7ff02819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 --2- 172.21.15.27:0/2175729951 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0280757b0 0x7ff02819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:05.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 >> 172.21.15.27:0/2175729951 conn(0x7ff0280ff5e0 msgr2=0x7ff028107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:05.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 shutdown_connections 2024-09-21T01:21:05.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:05.418+0000 7ff02ea00640 1 -- 172.21.15.27:0/2175729951 wait complete. 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:05 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2175729951' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:05 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2175729951' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T01:21:05.971 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:05.972 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:21:05.972 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.972 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.972 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.972 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:05.972 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:05 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2175729951' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T01:21:05.984 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790021 got 158913790024 for osd.5 2024-09-21T01:21:05.984 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:06.322 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593791 got 184683593794 for osd.6 2024-09-21T01:21:06.322 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:07.074 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149776 got 111669149778 for osd.3 2024-09-21T01:21:07.075 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:07.312 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:07.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.313 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:07 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:07 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.712 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678052 got 304942678055 for osd.11 2024-09-21T01:21:07.712 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:07.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:07 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:08.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542235 got 60129542237 for osd.1 2024-09-21T01:21:08.210 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:08.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705761 got 38654705764 for osd.0 2024-09-21T01:21:08.862 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:09.243 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:09.247 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:09.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430265 got 206158430268 for osd.7 2024-09-21T01:21:09.279 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:09.841 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:09 smithi027 ceph-mon[30375]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-21T01:21:09.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:09 smithi098 ceph-mon[34372]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-21T01:21:10.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:09 smithi089 ceph-mon[34486]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-21T01:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 -- 172.21.15.27:0/2073466034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210077940 msgr2=0x7f0210077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 --2- 172.21.15.27:0/2073466034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210077940 0x7f0210077d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0200009900 tx=0x7f020002a410 comp rx=0 tx=0).stop 2024-09-21T01:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 -- 172.21.15.27:0/2073466034 shutdown_connections 2024-09-21T01:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 --2- 172.21.15.27:0/2073466034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0210114580 0x7f02101169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 --2- 172.21.15.27:0/2073466034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210078290 0x7f0210114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 --2- 172.21.15.27:0/2073466034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210077940 0x7f0210077d50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 -- 172.21.15.27:0/2073466034 >> 172.21.15.27:0/2073466034 conn(0x7f0210101670 msgr2=0x7f0210103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 -- 172.21.15.27:0/2073466034 shutdown_connections 2024-09-21T01:21:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.157+0000 7f020f200640 1 -- 172.21.15.27:0/2073466034 wait complete. 2024-09-21T01:21:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.158+0000 7f020f200640 1 Processor -- start 2024-09-21T01:21:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.158+0000 7f020f200640 1 -- start start 2024-09-21T01:21:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210077940 0x7f02101a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 0x7f02101a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0210114580 0x7f02101a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0210119f30 con 0x7f0210078290 2024-09-21T01:21:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0210119db0 con 0x7f0210077940 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f021011a0b0 con 0x7f0210114580 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f0207e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 0x7f02101a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f0207e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 0x7f02101a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36724/0 (socket says 172.21.15.27:36724) 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f0207e00640 1 -- 172.21.15.27:0/841363622 learned_addr learned my addr 172.21.15.27:0/841363622 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020de00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210077940 0x7f02101a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.159+0000 7f020e800640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0210114580 0x7f02101a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0207e00640 1 -- 172.21.15.27:0/841363622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0210114580 msgr2=0x7f02101a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0207e00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0210114580 0x7f02101a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0207e00640 1 -- 172.21.15.27:0/841363622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210077940 msgr2=0x7f02101a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0207e00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210077940 0x7f02101a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0207e00640 1 -- 172.21.15.27:0/841363622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02101a68b0 con 0x7f0210078290 2024-09-21T01:21:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f020de00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210077940 0x7f02101a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:10.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0207e00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 0x7f02101a0ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f01f800e9f0 tx=0x7f01f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f800ccc0 con 0x7f0210078290 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f80108e0 con 0x7f0210078290 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02101a6ba0 con 0x7f0210078290 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.160+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8018860 con 0x7f0210078290 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.161+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101ad300 con 0x7f0210078290 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.162+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0210077d50 con 0x7f0210078290 2024-09-21T01:21:10.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.164+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f01f8010430 con 0x7f0210078290 2024-09-21T01:21:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.167+0000 7f0206a00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01f40b6670 0x7f01f40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.167+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f01f80e80e0 con 0x7f0210078290 2024-09-21T01:21:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.167+0000 7f020de00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01f40b6670 0x7f01f40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.168+0000 7f020de00640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01f40b6670 0x7f01f40b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0200005ab0 tx=0x7f0200005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:10.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.168+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01f80b2700 con 0x7f0210078290 2024-09-21T01:21:10.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.370+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f02101a1a70 con 0x7f0210078290 2024-09-21T01:21:10.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.371+0000 7f0206a00640 1 -- 172.21.15.27:0/841363622 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f02101a1a70 con 0x7f0210078290 2024-09-21T01:21:10.371 INFO:teuthology.orchestra.run.smithi027.stdout:257698037809 2024-09-21T01:21:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01f40b6670 msgr2=0x7f01f40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01f40b6670 0x7f01f40b8b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0200005ab0 tx=0x7f0200005a20 comp rx=0 tx=0).stop 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 msgr2=0x7f02101a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 0x7f02101a0ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f01f800e9f0 tx=0x7f01f800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 shutdown_connections 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01f40b6670 0x7f01f40b8b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0210114580 0x7f02101a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0210078290 0x7f02101a0ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 --2- 172.21.15.27:0/841363622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0210077940 0x7f02101a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.373+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 >> 172.21.15.27:0/841363622 conn(0x7f0210101670 msgr2=0x7f0210116610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.374+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 shutdown_connections 2024-09-21T01:21:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.374+0000 7f020f200640 1 -- 172.21.15.27:0/841363622 wait complete. 2024-09-21T01:21:10.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 -- 172.21.15.27:0/1630015873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a81047c0 msgr2=0x7f98a8104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 --2- 172.21.15.27:0/1630015873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a81047c0 0x7f98a8104bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9898009900 tx=0x7f989802a410 comp rx=0 tx=0).stop 2024-09-21T01:21:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 -- 172.21.15.27:0/1630015873 shutdown_connections 2024-09-21T01:21:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 --2- 172.21.15.27:0/1630015873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 0x7f98a8110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 --2- 172.21.15.27:0/1630015873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a8105110 0x7f98a810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 --2- 172.21.15.27:0/1630015873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a81047c0 0x7f98a8104bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 -- 172.21.15.27:0/1630015873 >> 172.21.15.27:0/1630015873 conn(0x7f98a80fe2c0 msgr2=0x7f98a8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 -- 172.21.15.27:0/1630015873 shutdown_connections 2024-09-21T01:21:10.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.512+0000 7f98a7200640 1 -- 172.21.15.27:0/1630015873 wait complete. 2024-09-21T01:21:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.513+0000 7f98a7200640 1 Processor -- start 2024-09-21T01:21:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 -- start start 2024-09-21T01:21:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 0x7f98a819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a8105110 0x7f98a819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 0x7f98a819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81138d0 con 0x7f98a8105110 2024-09-21T01:21:10.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8113750 con 0x7f98a810dc20 2024-09-21T01:21:10.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a7200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8113a50 con 0x7f98a81047c0 2024-09-21T01:21:10.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 0x7f98a819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.514+0000 7f98a5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a8105110 0x7f98a819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 0x7f98a819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39880/0 (socket says 172.21.15.27:39880) 2024-09-21T01:21:10.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 -- 172.21.15.27:0/3446038483 learned_addr learned my addr 172.21.15.27:0/3446038483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a6800640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 0x7f98a819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 -- 172.21.15.27:0/3446038483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 msgr2=0x7f98a819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 0x7f98a819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 -- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a8105110 msgr2=0x7f98a819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a8105110 0x7f98a819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 -- 172.21.15.27:0/3446038483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a81a0350 con 0x7f98a81047c0 2024-09-21T01:21:10.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a6800640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 0x7f98a819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5400640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a8105110 0x7f98a819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.515+0000 7f98a5e00640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 0x7f98a819a3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9898005ab0 tx=0x7f9898003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.517+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989803f070 con 0x7f98a81047c0 2024-09-21T01:21:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.517+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9898031b60 con 0x7f98a81047c0 2024-09-21T01:21:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.517+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a81a05e0 con 0x7f98a81047c0 2024-09-21T01:21:10.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.517+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898035030 con 0x7f98a81047c0 2024-09-21T01:21:10.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.517+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a81a6da0 con 0x7f98a81047c0 2024-09-21T01:21:10.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.518+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a8104bd0 con 0x7f98a81047c0 2024-09-21T01:21:10.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.520+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f9898042070 con 0x7f98a81047c0 2024-09-21T01:21:10.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.522+0000 7f989ea00640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f988c0b6670 0x7f988c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:10.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.522+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9898108080 con 0x7f98a81047c0 2024-09-21T01:21:10.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.522+0000 7f98a5400640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f988c0b6670 0x7f988c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:10.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.523+0000 7f98a5400640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f988c0b6670 0x7f988c0b8b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98a8110160 tx=0x7f9894005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:10.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.526+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98980ce2c0 con 0x7f98a81047c0 2024-09-21T01:21:10.724 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:10 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/841363622' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T01:21:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.729+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f98a81a6fd0 con 0x7f98a81047c0 2024-09-21T01:21:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.730+0000 7f989ea00640 1 -- 172.21.15.27:0/3446038483 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f98980d26a0 con 0x7f98a81047c0 2024-09-21T01:21:10.730 INFO:teuthology.orchestra.run.smithi027.stdout:279172874285 2024-09-21T01:21:10.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f988c0b6670 msgr2=0x7f988c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f988c0b6670 0x7f988c0b8b80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98a8110160 tx=0x7f9894005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:10.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 msgr2=0x7f98a819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 0x7f98a819a3e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9898005ab0 tx=0x7f9898003c10 comp rx=0 tx=0).stop 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 shutdown_connections 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f988c0b6670 0x7f988c0b8b80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f98a810dc20 0x7f98a819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98a8105110 0x7f98a819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 --2- 172.21.15.27:0/3446038483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98a81047c0 0x7f98a819a3e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 >> 172.21.15.27:0/3446038483 conn(0x7f98a80fe2c0 msgr2=0x7f98a8107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 shutdown_connections 2024-09-21T01:21:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:10.733+0000 7f98a7200640 1 -- 172.21.15.27:0/3446038483 wait complete. 2024-09-21T01:21:10.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:10 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/841363622' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T01:21:11.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:10 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/841363622' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T01:21:11.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:11 smithi027 ceph-mon[30375]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T01:21:11.866 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:11 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3446038483' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T01:21:11.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:11 smithi098 ceph-mon[34372]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T01:21:11.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:11 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3446038483' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T01:21:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:11 smithi089 ceph-mon[34486]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-21T01:21:12.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:11 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3446038483' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T01:21:12.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037805 got 257698037809 for osd.9 2024-09-21T01:21:12.280 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:12.673 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874282 got 279172874285 for osd.10 2024-09-21T01:21:12.674 DEBUG:teuthology.parallel:result is None 2024-09-21T01:21:12.674 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T01:21:12.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:21:13.109 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:13.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 -- 172.21.15.27:0/590722065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 msgr2=0x7f01a0077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 --2- 172.21.15.27:0/590722065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a0077d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0190009860 tx=0x7f019002a340 comp rx=0 tx=0).stop 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 -- 172.21.15.27:0/590722065 shutdown_connections 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 --2- 172.21.15.27:0/590722065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0114740 0x7f01a0116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 --2- 172.21.15.27:0/590722065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a0078290 0x7f01a0114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 --2- 172.21.15.27:0/590722065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a0077d50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 -- 172.21.15.27:0/590722065 >> 172.21.15.27:0/590722065 conn(0x7f01a01017d0 msgr2=0x7f01a0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 -- 172.21.15.27:0/590722065 shutdown_connections 2024-09-21T01:21:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.783+0000 7f01a5600640 1 -- 172.21.15.27:0/590722065 wait complete. 2024-09-21T01:21:13.785 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:13 smithi027 ceph-mon[30375]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.784+0000 7f01a5600640 1 Processor -- start 2024-09-21T01:21:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 -- start start 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 0x7f01a01a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a0114740 0x7f01a01a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01a011a0b0 con 0x7f01a0114740 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0119f30 con 0x7f01a0078290 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f01a5600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f01a011a230 con 0x7f01a0077940 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f019e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 0x7f01a01a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f019ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f019f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a0114740 0x7f01a01a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f019e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 0x7f01a01a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:41962/0 (socket says 172.21.15.27:41962) 2024-09-21T01:21:13.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.785+0000 7f019ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a01a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39904/0 (socket says 172.21.15.27:39904) 2024-09-21T01:21:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 -- 172.21.15.27:0/3213569786 learned_addr learned my addr 172.21.15.27:0/3213569786 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 -- 172.21.15.27:0/3213569786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 msgr2=0x7f01a01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a01a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 -- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a0114740 msgr2=0x7f01a01a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a0114740 0x7f01a01a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:13.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 -- 172.21.15.27:0/3213569786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01a6a70 con 0x7f01a0078290 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.786+0000 7f019e000640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 0x7f01a01a10a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f018c0058c0 tx=0x7f018c00ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.787+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c00cc90 con 0x7f01a0078290 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.787+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f018c00ce30 con 0x7f01a0078290 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.787+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a01a6d60 con 0x7f01a0078290 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.787+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c010430 con 0x7f01a0078290 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.787+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01ad4c0 con 0x7f01a0078290 2024-09-21T01:21:13.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.788+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a0077d50 con 0x7f01a0078290 2024-09-21T01:21:13.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.791+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f018c003710 con 0x7f01a0078290 2024-09-21T01:21:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.792+0000 7f0197400640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01840b65f0 0x7f01840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.792+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f018c0e7fb0 con 0x7f01a0078290 2024-09-21T01:21:13.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.793+0000 7f019ea00640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01840b65f0 0x7f01840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:13.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.793+0000 7f019ea00640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01840b65f0 0x7f01840b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0190009830 tx=0x7f0190030340 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:13.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:13.794+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f018c0ae1f0 con 0x7f01a0078290 2024-09-21T01:21:13.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:13 smithi098 ceph-mon[34372]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:14.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:13 smithi089 ceph-mon[34486]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.004+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01a0063800 con 0x7f01840b65f0 2024-09-21T01:21:14.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.012+0000 7f0197400640 1 -- 172.21.15.27:0/3213569786 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340418 (secure 0 0 0) 0x7f01a0063800 con 0x7f01840b65f0 2024-09-21T01:21:14.016 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:14.017 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01840b65f0 msgr2=0x7f01840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01840b65f0 0x7f01840b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0190009830 tx=0x7f0190030340 comp rx=0 tx=0).stop 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 msgr2=0x7f01a01a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 0x7f01a01a10a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f018c0058c0 tx=0x7f018c00ed90 comp rx=0 tx=0).stop 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 shutdown_connections 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f01840b65f0 0x7f01840b8b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a0114740 0x7f01a01a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01a0078290 0x7f01a01a10a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 --2- 172.21.15.27:0/3213569786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01a0077940 0x7f01a01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 >> 172.21.15.27:0/3213569786 conn(0x7f01a01017d0 msgr2=0x7f01a0116860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 shutdown_connections 2024-09-21T01:21:14.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:14.017+0000 7f01a5600640 1 -- 172.21.15.27:0/3213569786 wait complete. 2024-09-21T01:21:14.910 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-21T01:21:12.453974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664018,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":729,"num_read_kb":488,"num_write":375,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3019356,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372092,"kb_used_data":14444,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569732,"statfs":{"total":1151940427776,"available":1151559405568,"internally_reserved":0,"allocated":14790656,"data_stored":8385164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004064"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:08.500288+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:21:08.500288+0000","last_peered":"2024-09-21T01:21:08.500288+0000","last_clean":"2024-09-21T01:21:08.500288+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:21:08.500288+0000","last_undegraded":"2024-09-21T01:21:08.500288+0000","last_fullsized":"2024-09-21T01:21:08.500288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166928+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:20:42.166928+0000","last_peered":"2024-09-21T01:20:42.166928+0000","last_clean":"2024-09-21T01:20:42.166928+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:20:42.166928+0000","last_undegraded":"2024-09-21T01:20:42.166928+0000","last_fullsized":"2024-09-21T01:20:42.166928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166049+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:20:42.166049+0000","last_peered":"2024-09-21T01:20:42.166049+0000","last_clean":"2024-09-21T01:20:42.166049+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:20:42.166049+0000","last_undegraded":"2024-09-21T01:20:42.166049+0000","last_fullsized":"2024-09-21T01:20:42.166049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213160+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:20:42.213160+0000","last_peered":"2024-09-21T01:20:42.213160+0000","last_clean":"2024-09-21T01:20:42.213160+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:20:42.213160+0000","last_undegraded":"2024-09-21T01:20:42.213160+0000","last_fullsized":"2024-09-21T01:20:42.213160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171832+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:20:42.171832+0000","last_peered":"2024-09-21T01:20:42.171832+0000","last_clean":"2024-09-21T01:20:42.171832+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:20:42.171832+0000","last_undegraded":"2024-09-21T01:20:42.171832+0000","last_fullsized":"2024-09-21T01:20:42.171832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168709+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:20:42.168709+0000","last_peered":"2024-09-21T01:20:42.168709+0000","last_clean":"2024-09-21T01:20:42.168709+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:20:42.168709+0000","last_undegraded":"2024-09-21T01:20:42.168709+0000","last_fullsized":"2024-09-21T01:20:42.168709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172079+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:20:42.172079+0000","last_peered":"2024-09-21T01:20:42.172079+0000","last_clean":"2024-09-21T01:20:42.172079+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:20:42.172079+0000","last_undegraded":"2024-09-21T01:20:42.172079+0000","last_fullsized":"2024-09-21T01:20:42.172079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173133+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:20:42.173133+0000","last_peered":"2024-09-21T01:20:42.173133+0000","last_clean":"2024-09-21T01:20:42.173133+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:20:42.173133+0000","last_undegraded":"2024-09-21T01:20:42.173133+0000","last_fullsized":"2024-09-21T01:20:42.173133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169342+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:20:42.169342+0000","last_peered":"2024-09-21T01:20:42.169342+0000","last_clean":"2024-09-21T01:20:42.169342+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:20:42.169342+0000","last_undegraded":"2024-09-21T01:20:42.169342+0000","last_fullsized":"2024-09-21T01:20:42.169342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172395+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:42.172395+0000","last_peered":"2024-09-21T01:20:42.172395+0000","last_clean":"2024-09-21T01:20:42.172395+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:42.172395+0000","last_undegraded":"2024-09-21T01:20:42.172395+0000","last_fullsized":"2024-09-21T01:20:42.172395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174059+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:20:42.174059+0000","last_peered":"2024-09-21T01:20:42.174059+0000","last_clean":"2024-09-21T01:20:42.174059+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:20:42.174059+0000","last_undegraded":"2024-09-21T01:20:42.174059+0000","last_fullsized":"2024-09-21T01:20:42.174059+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169252+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:20:42.169252+0000","last_peered":"2024-09-21T01:20:42.169252+0000","last_clean":"2024-09-21T01:20:42.169252+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:20:42.169252+0000","last_undegraded":"2024-09-21T01:20:42.169252+0000","last_fullsized":"2024-09-21T01:20:42.169252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166612+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:20:42.166612+0000","last_peered":"2024-09-21T01:20:42.166612+0000","last_clean":"2024-09-21T01:20:42.166612+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:20:42.166612+0000","last_undegraded":"2024-09-21T01:20:42.166612+0000","last_fullsized":"2024-09-21T01:20:42.166612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171717+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:42.171717+0000","last_peered":"2024-09-21T01:20:42.171717+0000","last_clean":"2024-09-21T01:20:42.171717+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:42.171717+0000","last_undegraded":"2024-09-21T01:20:42.171717+0000","last_fullsized":"2024-09-21T01:20:42.171717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170083+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:20:42.170083+0000","last_peered":"2024-09-21T01:20:42.170083+0000","last_clean":"2024-09-21T01:20:42.170083+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:20:42.170083+0000","last_undegraded":"2024-09-21T01:20:42.170083+0000","last_fullsized":"2024-09-21T01:20:42.170083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212732+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:20:42.212732+0000","last_peered":"2024-09-21T01:20:42.212732+0000","last_clean":"2024-09-21T01:20:42.212732+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:20:42.212732+0000","last_undegraded":"2024-09-21T01:20:42.212732+0000","last_fullsized":"2024-09-21T01:20:42.212732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166193+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:42.166193+0000","last_peered":"2024-09-21T01:20:42.166193+0000","last_clean":"2024-09-21T01:20:42.166193+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:42.166193+0000","last_undegraded":"2024-09-21T01:20:42.166193+0000","last_fullsized":"2024-09-21T01:20:42.166193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169923+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:20:42.169923+0000","last_peered":"2024-09-21T01:20:42.169923+0000","last_clean":"2024-09-21T01:20:42.169923+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:20:42.169923+0000","last_undegraded":"2024-09-21T01:20:42.169923+0000","last_fullsized":"2024-09-21T01:20:42.169923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168722+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:20:42.168722+0000","last_peered":"2024-09-21T01:20:42.168722+0000","last_clean":"2024-09-21T01:20:42.168722+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:20:42.168722+0000","last_undegraded":"2024-09-21T01:20:42.168722+0000","last_fullsized":"2024-09-21T01:20:42.168722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212760+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:42.212760+0000","last_peered":"2024-09-21T01:20:42.212760+0000","last_clean":"2024-09-21T01:20:42.212760+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:42.212760+0000","last_undegraded":"2024-09-21T01:20:42.212760+0000","last_fullsized":"2024-09-21T01:20:42.212760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166926+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:20:42.166926+0000","last_peered":"2024-09-21T01:20:42.166926+0000","last_clean":"2024-09-21T01:20:42.166926+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:20:42.166926+0000","last_undegraded":"2024-09-21T01:20:42.166926+0000","last_fullsized":"2024-09-21T01:20:42.166926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171243+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:20:42.171243+0000","last_peered":"2024-09-21T01:20:42.171243+0000","last_clean":"2024-09-21T01:20:42.171243+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:20:42.171243+0000","last_undegraded":"2024-09-21T01:20:42.171243+0000","last_fullsized":"2024-09-21T01:20:42.171243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173345+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:42.173345+0000","last_peered":"2024-09-21T01:20:42.173345+0000","last_clean":"2024-09-21T01:20:42.173345+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:42.173345+0000","last_undegraded":"2024-09-21T01:20:42.173345+0000","last_fullsized":"2024-09-21T01:20:42.173345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170064+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:20:42.170064+0000","last_peered":"2024-09-21T01:20:42.170064+0000","last_clean":"2024-09-21T01:20:42.170064+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.170064+0000","last_undegraded":"2024-09-21T01:20:42.170064+0000","last_fullsized":"2024-09-21T01:20:42.170064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171766+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:20:42.171766+0000","last_peered":"2024-09-21T01:20:42.171766+0000","last_clean":"2024-09-21T01:20:42.171766+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:20:42.171766+0000","last_undegraded":"2024-09-21T01:20:42.171766+0000","last_fullsized":"2024-09-21T01:20:42.171766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170377+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:42.170377+0000","last_peered":"2024-09-21T01:20:42.170377+0000","last_clean":"2024-09-21T01:20:42.170377+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:42.170377+0000","last_undegraded":"2024-09-21T01:20:42.170377+0000","last_fullsized":"2024-09-21T01:20:42.170377+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166231+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:20:42.166231+0000","last_peered":"2024-09-21T01:20:42.166231+0000","last_clean":"2024-09-21T01:20:42.166231+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:20:42.166231+0000","last_undegraded":"2024-09-21T01:20:42.166231+0000","last_fullsized":"2024-09-21T01:20:42.166231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213147+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:20:42.213147+0000","last_peered":"2024-09-21T01:20:42.213147+0000","last_clean":"2024-09-21T01:20:42.213147+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:20:42.213147+0000","last_undegraded":"2024-09-21T01:20:42.213147+0000","last_fullsized":"2024-09-21T01:20:42.213147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'16","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167400+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:20:42.167400+0000","last_peered":"2024-09-21T01:20:42.167400+0000","last_clean":"2024-09-21T01:20:42.167400+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:20:42.167400+0000","last_undegraded":"2024-09-21T01:20:42.167400+0000","last_fullsized":"2024-09-21T01:20:42.167400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212422+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:20:42.212422+0000","last_peered":"2024-09-21T01:20:42.212422+0000","last_clean":"2024-09-21T01:20:42.212422+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.212422+0000","last_undegraded":"2024-09-21T01:20:42.212422+0000","last_fullsized":"2024-09-21T01:20:42.212422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368591+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:20:42.368591+0000","last_peered":"2024-09-21T01:20:42.368591+0000","last_clean":"2024-09-21T01:20:42.368591+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:20:42.368591+0000","last_undegraded":"2024-09-21T01:20:42.368591+0000","last_fullsized":"2024-09-21T01:20:42.368591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.185170+0000","last_change":"2024-09-21T01:20:42.184446+0000","last_active":"2024-09-21T01:20:42.185170+0000","last_peered":"2024-09-21T01:20:42.185170+0000","last_clean":"2024-09-21T01:20:42.185170+0000","last_became_active":"2024-09-21T01:20:42.172338+0000","last_became_peered":"2024-09-21T01:20:42.172338+0000","last_unstale":"2024-09-21T01:20:42.185170+0000","last_undegraded":"2024-09-21T01:20:42.185170+0000","last_fullsized":"2024-09-21T01:20:42.185170+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:12.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168686+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:20:42.168686+0000","last_peered":"2024-09-21T01:20:42.168686+0000","last_clean":"2024-09-21T01:20:42.168686+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:20:42.168686+0000","last_undegraded":"2024-09-21T01:20:42.168686+0000","last_fullsized":"2024-09-21T01:20:42.168686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169233+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:20:42.169233+0000","last_peered":"2024-09-21T01:20:42.169233+0000","last_clean":"2024-09-21T01:20:42.169233+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:20:42.169233+0000","last_undegraded":"2024-09-21T01:20:42.169233+0000","last_fullsized":"2024-09-21T01:20:42.169233+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167193+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:20:42.167193+0000","last_peered":"2024-09-21T01:20:42.167193+0000","last_clean":"2024-09-21T01:20:42.167193+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:20:42.167193+0000","last_undegraded":"2024-09-21T01:20:42.167193+0000","last_fullsized":"2024-09-21T01:20:42.167193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171149+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:20:42.171149+0000","last_peered":"2024-09-21T01:20:42.171149+0000","last_clean":"2024-09-21T01:20:42.171149+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:20:42.171149+0000","last_undegraded":"2024-09-21T01:20:42.171149+0000","last_fullsized":"2024-09-21T01:20:42.171149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168900+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:20:42.168900+0000","last_peered":"2024-09-21T01:20:42.168900+0000","last_clean":"2024-09-21T01:20:42.168900+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:20:42.168900+0000","last_undegraded":"2024-09-21T01:20:42.168900+0000","last_fullsized":"2024-09-21T01:20:42.168900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:09.502229+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:21:09.502229+0000","last_peered":"2024-09-21T01:21:09.502229+0000","last_clean":"2024-09-21T01:21:09.502229+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:21:09.502229+0000","last_undegraded":"2024-09-21T01:21:09.502229+0000","last_fullsized":"2024-09-21T01:21:09.502229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368463+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:20:42.368463+0000","last_peered":"2024-09-21T01:20:42.368463+0000","last_clean":"2024-09-21T01:20:42.368463+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:20:42.368463+0000","last_undegraded":"2024-09-21T01:20:42.368463+0000","last_fullsized":"2024-09-21T01:20:42.368463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173332+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:20:42.173332+0000","last_peered":"2024-09-21T01:20:42.173332+0000","last_clean":"2024-09-21T01:20:42.173332+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:20:42.173332+0000","last_undegraded":"2024-09-21T01:20:42.173332+0000","last_fullsized":"2024-09-21T01:20:42.173332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:08.504293+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:21:08.504293+0000","last_peered":"2024-09-21T01:21:08.504293+0000","last_clean":"2024-09-21T01:21:08.504293+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:21:08.504293+0000","last_undegraded":"2024-09-21T01:21:08.504293+0000","last_fullsized":"2024-09-21T01:21:08.504293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368606+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:20:42.368606+0000","last_peered":"2024-09-21T01:20:42.368606+0000","last_clean":"2024-09-21T01:20:42.368606+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:20:42.368606+0000","last_undegraded":"2024-09-21T01:20:42.368606+0000","last_fullsized":"2024-09-21T01:20:42.368606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368095+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:20:42.368095+0000","last_peered":"2024-09-21T01:20:42.368095+0000","last_clean":"2024-09-21T01:20:42.368095+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:20:42.368095+0000","last_undegraded":"2024-09-21T01:20:42.368095+0000","last_fullsized":"2024-09-21T01:20:42.368095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166486+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:42.166486+0000","last_peered":"2024-09-21T01:20:42.166486+0000","last_clean":"2024-09-21T01:20:42.166486+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:42.166486+0000","last_undegraded":"2024-09-21T01:20:42.166486+0000","last_fullsized":"2024-09-21T01:20:42.166486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171822+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:20:42.171822+0000","last_peered":"2024-09-21T01:20:42.171822+0000","last_clean":"2024-09-21T01:20:42.171822+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:20:42.171822+0000","last_undegraded":"2024-09-21T01:20:42.171822+0000","last_fullsized":"2024-09-21T01:20:42.171822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166644+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:20:42.166644+0000","last_peered":"2024-09-21T01:20:42.166644+0000","last_clean":"2024-09-21T01:20:42.166644+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:20:42.166644+0000","last_undegraded":"2024-09-21T01:20:42.166644+0000","last_fullsized":"2024-09-21T01:20:42.166644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169791+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:42.169791+0000","last_peered":"2024-09-21T01:20:42.169791+0000","last_clean":"2024-09-21T01:20:42.169791+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:42.169791+0000","last_undegraded":"2024-09-21T01:20:42.169791+0000","last_fullsized":"2024-09-21T01:20:42.169791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166597+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:20:42.166597+0000","last_peered":"2024-09-21T01:20:42.166597+0000","last_clean":"2024-09-21T01:20:42.166597+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:20:42.166597+0000","last_undegraded":"2024-09-21T01:20:42.166597+0000","last_fullsized":"2024-09-21T01:20:42.166597+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166585+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:20:42.166585+0000","last_peered":"2024-09-21T01:20:42.166585+0000","last_clean":"2024-09-21T01:20:42.166585+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:20:42.166585+0000","last_undegraded":"2024-09-21T01:20:42.166585+0000","last_fullsized":"2024-09-21T01:20:42.166585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166859+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:20:42.166859+0000","last_peered":"2024-09-21T01:20:42.166859+0000","last_clean":"2024-09-21T01:20:42.166859+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:20:42.166859+0000","last_undegraded":"2024-09-21T01:20:42.166859+0000","last_fullsized":"2024-09-21T01:20:42.166859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:08.226573+0000","last_change":"2024-09-21T01:20:42.188731+0000","last_active":"2024-09-21T01:21:08.226573+0000","last_peered":"2024-09-21T01:21:08.226573+0000","last_clean":"2024-09-21T01:21:08.226573+0000","last_became_active":"2024-09-21T01:20:42.172041+0000","last_became_peered":"2024-09-21T01:20:42.172041+0000","last_unstale":"2024-09-21T01:21:08.226573+0000","last_undegraded":"2024-09-21T01:21:08.226573+0000","last_fullsized":"2024-09-21T01:21:08.226573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:42.392074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173015+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:20:42.173015+0000","last_peered":"2024-09-21T01:20:42.173015+0000","last_clean":"2024-09-21T01:20:42.173015+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:20:42.173015+0000","last_undegraded":"2024-09-21T01:20:42.173015+0000","last_fullsized":"2024-09-21T01:20:42.173015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166727+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:20:42.166727+0000","last_peered":"2024-09-21T01:20:42.166727+0000","last_clean":"2024-09-21T01:20:42.166727+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:20:42.166727+0000","last_undegraded":"2024-09-21T01:20:42.166727+0000","last_fullsized":"2024-09-21T01:20:42.166727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174353+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:20:42.174353+0000","last_peered":"2024-09-21T01:20:42.174353+0000","last_clean":"2024-09-21T01:20:42.174353+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:20:42.174353+0000","last_undegraded":"2024-09-21T01:20:42.174353+0000","last_fullsized":"2024-09-21T01:20:42.174353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171285+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:20:42.171285+0000","last_peered":"2024-09-21T01:20:42.171285+0000","last_clean":"2024-09-21T01:20:42.171285+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:20:42.171285+0000","last_undegraded":"2024-09-21T01:20:42.171285+0000","last_fullsized":"2024-09-21T01:20:42.171285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167009+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:20:42.167009+0000","last_peered":"2024-09-21T01:20:42.167009+0000","last_clean":"2024-09-21T01:20:42.167009+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:20:42.167009+0000","last_undegraded":"2024-09-21T01:20:42.167009+0000","last_fullsized":"2024-09-21T01:20:42.167009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368627+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:42.368627+0000","last_peered":"2024-09-21T01:20:42.368627+0000","last_clean":"2024-09-21T01:20:42.368627+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:42.368627+0000","last_undegraded":"2024-09-21T01:20:42.368627+0000","last_fullsized":"2024-09-21T01:20:42.368627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173098+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:20:42.173098+0000","last_peered":"2024-09-21T01:20:42.173098+0000","last_clean":"2024-09-21T01:20:42.173098+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:20:42.173098+0000","last_undegraded":"2024-09-21T01:20:42.173098+0000","last_fullsized":"2024-09-21T01:20:42.173098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169084+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:20:42.169084+0000","last_peered":"2024-09-21T01:20:42.169084+0000","last_clean":"2024-09-21T01:20:42.169084+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:20:42.169084+0000","last_undegraded":"2024-09-21T01:20:42.169084+0000","last_fullsized":"2024-09-21T01:20:42.169084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.189333+0000","last_change":"2024-09-21T01:20:42.188808+0000","last_active":"2024-09-21T01:20:42.189333+0000","last_peered":"2024-09-21T01:20:42.189333+0000","last_clean":"2024-09-21T01:20:42.189333+0000","last_became_active":"2024-09-21T01:20:42.175348+0000","last_became_peered":"2024-09-21T01:20:42.175348+0000","last_unstale":"2024-09-21T01:20:42.189333+0000","last_undegraded":"2024-09-21T01:20:42.189333+0000","last_fullsized":"2024-09-21T01:20:42.189333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.746569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173522+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:20:42.173522+0000","last_peered":"2024-09-21T01:20:42.173522+0000","last_clean":"2024-09-21T01:20:42.173522+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:20:42.173522+0000","last_undegraded":"2024-09-21T01:20:42.173522+0000","last_fullsized":"2024-09-21T01:20:42.173522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173643+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:20:42.173643+0000","last_peered":"2024-09-21T01:20:42.173643+0000","last_clean":"2024-09-21T01:20:42.173643+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:20:42.173643+0000","last_undegraded":"2024-09-21T01:20:42.173643+0000","last_fullsized":"2024-09-21T01:20:42.173643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171162+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:42.171162+0000","last_peered":"2024-09-21T01:20:42.171162+0000","last_clean":"2024-09-21T01:20:42.171162+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:42.171162+0000","last_undegraded":"2024-09-21T01:20:42.171162+0000","last_fullsized":"2024-09-21T01:20:42.171162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368583+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:20:42.368583+0000","last_peered":"2024-09-21T01:20:42.368583+0000","last_clean":"2024-09-21T01:20:42.368583+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:20:42.368583+0000","last_undegraded":"2024-09-21T01:20:42.368583+0000","last_fullsized":"2024-09-21T01:20:42.368583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166303+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:20:42.166303+0000","last_peered":"2024-09-21T01:20:42.166303+0000","last_clean":"2024-09-21T01:20:42.166303+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:20:42.166303+0000","last_undegraded":"2024-09-21T01:20:42.166303+0000","last_fullsized":"2024-09-21T01:20:42.166303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:08.504880+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:21:08.504880+0000","last_peered":"2024-09-21T01:21:08.504880+0000","last_clean":"2024-09-21T01:21:08.504880+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:21:08.504880+0000","last_undegraded":"2024-09-21T01:21:08.504880+0000","last_fullsized":"2024-09-21T01:21:08.504880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172538+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:20:42.172538+0000","last_peered":"2024-09-21T01:20:42.172538+0000","last_clean":"2024-09-21T01:20:42.172538+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:20:42.172538+0000","last_undegraded":"2024-09-21T01:20:42.172538+0000","last_fullsized":"2024-09-21T01:20:42.172538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166615+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:20:42.166615+0000","last_peered":"2024-09-21T01:20:42.166615+0000","last_clean":"2024-09-21T01:20:42.166615+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:20:42.166615+0000","last_undegraded":"2024-09-21T01:20:42.166615+0000","last_fullsized":"2024-09-21T01:20:42.166615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173908+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:42.173908+0000","last_peered":"2024-09-21T01:20:42.173908+0000","last_clean":"2024-09-21T01:20:42.173908+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:42.173908+0000","last_undegraded":"2024-09-21T01:20:42.173908+0000","last_fullsized":"2024-09-21T01:20:42.173908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168692+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:20:42.168692+0000","last_peered":"2024-09-21T01:20:42.168692+0000","last_clean":"2024-09-21T01:20:42.168692+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:20:42.168692+0000","last_undegraded":"2024-09-21T01:20:42.168692+0000","last_fullsized":"2024-09-21T01:20:42.168692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368243+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:20:42.368243+0000","last_peered":"2024-09-21T01:20:42.368243+0000","last_clean":"2024-09-21T01:20:42.368243+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:20:42.368243+0000","last_undegraded":"2024-09-21T01:20:42.368243+0000","last_fullsized":"2024-09-21T01:20:42.368243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166554+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:42.166554+0000","last_peered":"2024-09-21T01:20:42.166554+0000","last_clean":"2024-09-21T01:20:42.166554+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:42.166554+0000","last_undegraded":"2024-09-21T01:20:42.166554+0000","last_fullsized":"2024-09-21T01:20:42.166554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173913+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:20:42.173913+0000","last_peered":"2024-09-21T01:20:42.173913+0000","last_clean":"2024-09-21T01:20:42.173913+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:20:42.173913+0000","last_undegraded":"2024-09-21T01:20:42.173913+0000","last_fullsized":"2024-09-21T01:20:42.173913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171904+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:20:42.171904+0000","last_peered":"2024-09-21T01:20:42.171904+0000","last_clean":"2024-09-21T01:20:42.171904+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:20:42.171904+0000","last_undegraded":"2024-09-21T01:20:42.171904+0000","last_fullsized":"2024-09-21T01:20:42.171904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:10.218958+0000","last_change":"2024-09-21T01:20:42.182111+0000","last_active":"2024-09-21T01:21:10.218958+0000","last_peered":"2024-09-21T01:21:10.218958+0000","last_clean":"2024-09-21T01:21:10.218958+0000","last_became_active":"2024-09-21T01:20:42.174484+0000","last_became_peered":"2024-09-21T01:20:42.174484+0000","last_unstale":"2024-09-21T01:21:10.218958+0000","last_undegraded":"2024-09-21T01:21:10.218958+0000","last_fullsized":"2024-09-21T01:21:10.218958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:45.448208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66212,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":133,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":66212,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166800+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:20:42.166800+0000","last_peered":"2024-09-21T01:20:42.166800+0000","last_clean":"2024-09-21T01:20:42.166800+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:20:42.166800+0000","last_undegraded":"2024-09-21T01:20:42.166800+0000","last_fullsized":"2024-09-21T01:20:42.166800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173328+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:20:42.173328+0000","last_peered":"2024-09-21T01:20:42.173328+0000","last_clean":"2024-09-21T01:20:42.173328+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:20:42.173328+0000","last_undegraded":"2024-09-21T01:20:42.173328+0000","last_fullsized":"2024-09-21T01:20:42.173328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174235+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:42.174235+0000","last_peered":"2024-09-21T01:20:42.174235+0000","last_clean":"2024-09-21T01:20:42.174235+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:42.174235+0000","last_undegraded":"2024-09-21T01:20:42.174235+0000","last_fullsized":"2024-09-21T01:20:42.174235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169546+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:20:42.169546+0000","last_peered":"2024-09-21T01:20:42.169546+0000","last_clean":"2024-09-21T01:20:42.169546+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:20:42.169546+0000","last_undegraded":"2024-09-21T01:20:42.169546+0000","last_fullsized":"2024-09-21T01:20:42.169546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171822+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:20:42.171822+0000","last_peered":"2024-09-21T01:20:42.171822+0000","last_clean":"2024-09-21T01:20:42.171822+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:20:42.171822+0000","last_undegraded":"2024-09-21T01:20:42.171822+0000","last_fullsized":"2024-09-21T01:20:42.171822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167088+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:42.167088+0000","last_peered":"2024-09-21T01:20:42.167088+0000","last_clean":"2024-09-21T01:20:42.167088+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:42.167088+0000","last_undegraded":"2024-09-21T01:20:42.167088+0000","last_fullsized":"2024-09-21T01:20:42.167088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166526+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:20:42.166526+0000","last_peered":"2024-09-21T01:20:42.166526+0000","last_clean":"2024-09-21T01:20:42.166526+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:20:42.166526+0000","last_undegraded":"2024-09-21T01:20:42.166526+0000","last_fullsized":"2024-09-21T01:20:42.166526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173219+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:20:42.173219+0000","last_peered":"2024-09-21T01:20:42.173219+0000","last_clean":"2024-09-21T01:20:42.173219+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:20:42.173219+0000","last_undegraded":"2024-09-21T01:20:42.173219+0000","last_fullsized":"2024-09-21T01:20:42.173219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:09.502242+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:21:09.502242+0000","last_peered":"2024-09-21T01:21:09.502242+0000","last_clean":"2024-09-21T01:21:09.502242+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:21:09.502242+0000","last_undegraded":"2024-09-21T01:21:09.502242+0000","last_fullsized":"2024-09-21T01:21:09.502242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212527+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:20:42.212527+0000","last_peered":"2024-09-21T01:20:42.212527+0000","last_clean":"2024-09-21T01:20:42.212527+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:20:42.212527+0000","last_undegraded":"2024-09-21T01:20:42.212527+0000","last_fullsized":"2024-09-21T01:20:42.212527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171628+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:20:42.171628+0000","last_peered":"2024-09-21T01:20:42.171628+0000","last_clean":"2024-09-21T01:20:42.171628+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:20:42.171628+0000","last_undegraded":"2024-09-21T01:20:42.171628+0000","last_fullsized":"2024-09-21T01:20:42.171628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166658+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:20:42.166658+0000","last_peered":"2024-09-21T01:20:42.166658+0000","last_clean":"2024-09-21T01:20:42.166658+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:20:42.166658+0000","last_undegraded":"2024-09-21T01:20:42.166658+0000","last_fullsized":"2024-09-21T01:20:42.166658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173384+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:42.173384+0000","last_peered":"2024-09-21T01:20:42.173384+0000","last_clean":"2024-09-21T01:20:42.173384+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:42.173384+0000","last_undegraded":"2024-09-21T01:20:42.173384+0000","last_fullsized":"2024-09-21T01:20:42.173384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169092+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:20:42.169092+0000","last_peered":"2024-09-21T01:20:42.169092+0000","last_clean":"2024-09-21T01:20:42.169092+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:20:42.169092+0000","last_undegraded":"2024-09-21T01:20:42.169092+0000","last_fullsized":"2024-09-21T01:20:42.169092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212586+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:20:42.212586+0000","last_peered":"2024-09-21T01:20:42.212586+0000","last_clean":"2024-09-21T01:20:42.212586+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:20:42.212586+0000","last_undegraded":"2024-09-21T01:20:42.212586+0000","last_fullsized":"2024-09-21T01:20:42.212586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368099+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:42.368099+0000","last_peered":"2024-09-21T01:20:42.368099+0000","last_clean":"2024-09-21T01:20:42.368099+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:42.368099+0000","last_undegraded":"2024-09-21T01:20:42.368099+0000","last_fullsized":"2024-09-21T01:20:42.368099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169896+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:42.169896+0000","last_peered":"2024-09-21T01:20:42.169896+0000","last_clean":"2024-09-21T01:20:42.169896+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:42.169896+0000","last_undegraded":"2024-09-21T01:20:42.169896+0000","last_fullsized":"2024-09-21T01:20:42.169896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173888+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:20:42.173888+0000","last_peered":"2024-09-21T01:20:42.173888+0000","last_clean":"2024-09-21T01:20:42.173888+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:20:42.173888+0000","last_undegraded":"2024-09-21T01:20:42.173888+0000","last_fullsized":"2024-09-21T01:20:42.173888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166218+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:20:42.166218+0000","last_peered":"2024-09-21T01:20:42.166218+0000","last_clean":"2024-09-21T01:20:42.166218+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:20:42.166218+0000","last_undegraded":"2024-09-21T01:20:42.166218+0000","last_fullsized":"2024-09-21T01:20:42.166218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:10.504491+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:21:10.504491+0000","last_peered":"2024-09-21T01:21:10.504491+0000","last_clean":"2024-09-21T01:21:10.504491+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:21:10.504491+0000","last_undegraded":"2024-09-21T01:21:10.504491+0000","last_fullsized":"2024-09-21T01:21:10.504491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171380+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:20:42.171380+0000","last_peered":"2024-09-21T01:20:42.171380+0000","last_clean":"2024-09-21T01:20:42.171380+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:20:42.171380+0000","last_undegraded":"2024-09-21T01:20:42.171380+0000","last_fullsized":"2024-09-21T01:20:42.171380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166508+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:20:42.166508+0000","last_peered":"2024-09-21T01:20:42.166508+0000","last_clean":"2024-09-21T01:20:42.166508+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:20:42.166508+0000","last_undegraded":"2024-09-21T01:20:42.166508+0000","last_fullsized":"2024-09-21T01:20:42.166508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168672+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:20:42.168672+0000","last_peered":"2024-09-21T01:20:42.168672+0000","last_clean":"2024-09-21T01:20:42.168672+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:20:42.168672+0000","last_undegraded":"2024-09-21T01:20:42.168672+0000","last_fullsized":"2024-09-21T01:20:42.168672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166554+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:20:42.166554+0000","last_peered":"2024-09-21T01:20:42.166554+0000","last_clean":"2024-09-21T01:20:42.166554+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:20:42.166554+0000","last_undegraded":"2024-09-21T01:20:42.166554+0000","last_fullsized":"2024-09-21T01:20:42.166554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166606+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:20:42.166606+0000","last_peered":"2024-09-21T01:20:42.166606+0000","last_clean":"2024-09-21T01:20:42.166606+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:20:42.166606+0000","last_undegraded":"2024-09-21T01:20:42.166606+0000","last_fullsized":"2024-09-21T01:20:42.166606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171403+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:20:42.171403+0000","last_peered":"2024-09-21T01:20:42.171403+0000","last_clean":"2024-09-21T01:20:42.171403+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:20:42.171403+0000","last_undegraded":"2024-09-21T01:20:42.171403+0000","last_fullsized":"2024-09-21T01:20:42.171403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368601+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:20:42.368601+0000","last_peered":"2024-09-21T01:20:42.368601+0000","last_clean":"2024-09-21T01:20:42.368601+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:20:42.368601+0000","last_undegraded":"2024-09-21T01:20:42.368601+0000","last_fullsized":"2024-09-21T01:20:42.368601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169605+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:20:42.169605+0000","last_peered":"2024-09-21T01:20:42.169605+0000","last_clean":"2024-09-21T01:20:42.169605+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:20:42.169605+0000","last_undegraded":"2024-09-21T01:20:42.169605+0000","last_fullsized":"2024-09-21T01:20:42.169605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166579+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:20:42.166579+0000","last_peered":"2024-09-21T01:20:42.166579+0000","last_clean":"2024-09-21T01:20:42.166579+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:20:42.166579+0000","last_undegraded":"2024-09-21T01:20:42.166579+0000","last_fullsized":"2024-09-21T01:20:42.166579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213117+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:20:42.213117+0000","last_peered":"2024-09-21T01:20:42.213117+0000","last_clean":"2024-09-21T01:20:42.213117+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:20:42.213117+0000","last_undegraded":"2024-09-21T01:20:42.213117+0000","last_fullsized":"2024-09-21T01:20:42.213117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173192+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:20:42.173192+0000","last_peered":"2024-09-21T01:20:42.173192+0000","last_clean":"2024-09-21T01:20:42.173192+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:20:42.173192+0000","last_undegraded":"2024-09-21T01:20:42.173192+0000","last_fullsized":"2024-09-21T01:20:42.173192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212633+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:20:42.212633+0000","last_peered":"2024-09-21T01:20:42.212633+0000","last_clean":"2024-09-21T01:20:42.212633+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:20:42.212633+0000","last_undegraded":"2024-09-21T01:20:42.212633+0000","last_fullsized":"2024-09-21T01:20:42.212633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173219+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:20:42.173219+0000","last_peered":"2024-09-21T01:20:42.173219+0000","last_clean":"2024-09-21T01:20:42.173219+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:20:42.173219+0000","last_undegraded":"2024-09-21T01:20:42.173219+0000","last_fullsized":"2024-09-21T01:20:42.173219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368426+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:20:42.368426+0000","last_peered":"2024-09-21T01:20:42.368426+0000","last_clean":"2024-09-21T01:20:42.368426+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:20:42.368426+0000","last_undegraded":"2024-09-21T01:20:42.368426+0000","last_fullsized":"2024-09-21T01:20:42.368426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168697+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:20:42.168697+0000","last_peered":"2024-09-21T01:20:42.168697+0000","last_clean":"2024-09-21T01:20:42.168697+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:20:42.168697+0000","last_undegraded":"2024-09-21T01:20:42.168697+0000","last_fullsized":"2024-09-21T01:20:42.168697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166639+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:20:42.166639+0000","last_peered":"2024-09-21T01:20:42.166639+0000","last_clean":"2024-09-21T01:20:42.166639+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:20:42.166639+0000","last_undegraded":"2024-09-21T01:20:42.166639+0000","last_fullsized":"2024-09-21T01:20:42.166639+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167341+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:20:42.167341+0000","last_peered":"2024-09-21T01:20:42.167341+0000","last_clean":"2024-09-21T01:20:42.167341+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:20:42.167341+0000","last_undegraded":"2024-09-21T01:20:42.167341+0000","last_fullsized":"2024-09-21T01:20:42.167341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169662+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:20:42.169662+0000","last_peered":"2024-09-21T01:20:42.169662+0000","last_clean":"2024-09-21T01:20:42.169662+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:20:42.169662+0000","last_undegraded":"2024-09-21T01:20:42.169662+0000","last_fullsized":"2024-09-21T01:20:42.169662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172681+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:20:42.172681+0000","last_peered":"2024-09-21T01:20:42.172681+0000","last_clean":"2024-09-21T01:20:42.172681+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:20:42.172681+0000","last_undegraded":"2024-09-21T01:20:42.172681+0000","last_fullsized":"2024-09-21T01:20:42.172681+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171018+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:20:42.171018+0000","last_peered":"2024-09-21T01:20:42.171018+0000","last_clean":"2024-09-21T01:20:42.171018+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:20:42.171018+0000","last_undegraded":"2024-09-21T01:20:42.171018+0000","last_fullsized":"2024-09-21T01:20:42.171018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171106+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:20:42.171106+0000","last_peered":"2024-09-21T01:20:42.171106+0000","last_clean":"2024-09-21T01:20:42.171106+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:20:42.171106+0000","last_undegraded":"2024-09-21T01:20:42.171106+0000","last_fullsized":"2024-09-21T01:20:42.171106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368338+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:20:42.368338+0000","last_peered":"2024-09-21T01:20:42.368338+0000","last_clean":"2024-09-21T01:20:42.368338+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:20:42.368338+0000","last_undegraded":"2024-09-21T01:20:42.368338+0000","last_fullsized":"2024-09-21T01:20:42.368338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170161+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:20:42.170161+0000","last_peered":"2024-09-21T01:20:42.170161+0000","last_clean":"2024-09-21T01:20:42.170161+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:20:42.170161+0000","last_undegraded":"2024-09-21T01:20:42.170161+0000","last_fullsized":"2024-09-21T01:20:42.170161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166766+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:20:42.166766+0000","last_peered":"2024-09-21T01:20:42.166766+0000","last_clean":"2024-09-21T01:20:42.166766+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:20:42.166766+0000","last_undegraded":"2024-09-21T01:20:42.166766+0000","last_fullsized":"2024-09-21T01:20:42.166766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170044+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:20:42.170044+0000","last_peered":"2024-09-21T01:20:42.170044+0000","last_clean":"2024-09-21T01:20:42.170044+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:20:42.170044+0000","last_undegraded":"2024-09-21T01:20:42.170044+0000","last_fullsized":"2024-09-21T01:20:42.170044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173327+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:20:42.173327+0000","last_peered":"2024-09-21T01:20:42.173327+0000","last_clean":"2024-09-21T01:20:42.173327+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:20:42.173327+0000","last_undegraded":"2024-09-21T01:20:42.173327+0000","last_fullsized":"2024-09-21T01:20:42.173327+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212552+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:20:42.212552+0000","last_peered":"2024-09-21T01:20:42.212552+0000","last_clean":"2024-09-21T01:20:42.212552+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:20:42.212552+0000","last_undegraded":"2024-09-21T01:20:42.212552+0000","last_fullsized":"2024-09-21T01:20:42.212552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166340+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:20:42.166340+0000","last_peered":"2024-09-21T01:20:42.166340+0000","last_clean":"2024-09-21T01:20:42.166340+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:20:42.166340+0000","last_undegraded":"2024-09-21T01:20:42.166340+0000","last_fullsized":"2024-09-21T01:20:42.166340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169730+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:20:42.169730+0000","last_peered":"2024-09-21T01:20:42.169730+0000","last_clean":"2024-09-21T01:20:42.169730+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:20:42.169730+0000","last_undegraded":"2024-09-21T01:20:42.169730+0000","last_fullsized":"2024-09-21T01:20:42.169730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171574+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:20:42.171574+0000","last_peered":"2024-09-21T01:20:42.171574+0000","last_clean":"2024-09-21T01:20:42.171574+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:20:42.171574+0000","last_undegraded":"2024-09-21T01:20:42.171574+0000","last_fullsized":"2024-09-21T01:20:42.171574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170266+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:20:42.170266+0000","last_peered":"2024-09-21T01:20:42.170266+0000","last_clean":"2024-09-21T01:20:42.170266+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:20:42.170266+0000","last_undegraded":"2024-09-21T01:20:42.170266+0000","last_fullsized":"2024-09-21T01:20:42.170266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171568+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:20:42.171568+0000","last_peered":"2024-09-21T01:20:42.171568+0000","last_clean":"2024-09-21T01:20:42.171568+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:20:42.171568+0000","last_undegraded":"2024-09-21T01:20:42.171568+0000","last_fullsized":"2024-09-21T01:20:42.171568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171626+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:20:42.171626+0000","last_peered":"2024-09-21T01:20:42.171626+0000","last_clean":"2024-09-21T01:20:42.171626+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:20:42.171626+0000","last_undegraded":"2024-09-21T01:20:42.171626+0000","last_fullsized":"2024-09-21T01:20:42.171626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169506+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:20:42.169506+0000","last_peered":"2024-09-21T01:20:42.169506+0000","last_clean":"2024-09-21T01:20:42.169506+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:20:42.169506+0000","last_undegraded":"2024-09-21T01:20:42.169506+0000","last_fullsized":"2024-09-21T01:20:42.169506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73416,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":648,"num_read_kb":419,"num_write":250,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":67516,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":220248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678056,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":535509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1753088,"data_stored":1192890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037810,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1073152,"data_stored":535305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":535549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712332,"statfs":{"total":95995035648,"available":95961427968,"internally_reserved":0,"allocated":1667072,"data_stored":1132571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":530809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1675264,"data_stored":1127762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1142784,"data_stored":602656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":530517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378715,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":531280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542240,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":598051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705766,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":532265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:21:14.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:21:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:14 smithi027 ceph-mon[30375]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:21:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:21:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:21:15.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:14 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:14 smithi089 ceph-mon[34486]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:21:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:21:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:21:15.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:14 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.339 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:14 smithi098 ceph-mon[34372]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:21:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:21:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:21:15.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:14 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.947+0000 7f360be00640 1 -- 172.21.15.27:0/1633816602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36041049b0 msgr2=0x7f3604104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.947+0000 7f360be00640 1 --2- 172.21.15.27:0/1633816602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36041049b0 0x7f3604104dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009900 tx=0x7f35f802a410 comp rx=0 tx=0).stop 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 -- 172.21.15.27:0/1633816602 shutdown_connections 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 --2- 172.21.15.27:0/1633816602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f360410de10 0x7f3604110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 --2- 172.21.15.27:0/1633816602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3604105300 0x7f360410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 --2- 172.21.15.27:0/1633816602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36041049b0 0x7f3604104dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 -- 172.21.15.27:0/1633816602 >> 172.21.15.27:0/1633816602 conn(0x7f3604079760 msgr2=0x7f3604079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 -- 172.21.15.27:0/1633816602 shutdown_connections 2024-09-21T01:21:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.948+0000 7f360be00640 1 -- 172.21.15.27:0/1633816602 wait complete. 2024-09-21T01:21:15.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.949+0000 7f360be00640 1 Processor -- start 2024-09-21T01:21:15.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.949+0000 7f360be00640 1 -- start start 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.949+0000 7f360be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 0x7f36041026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.949+0000 7f360be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3604105300 0x7f3604102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f360be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f360410de10 0x7f3604103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f360be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3604113a90 con 0x7f360410de10 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f360be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3604113910 con 0x7f3604105300 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f360be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3604113c10 con 0x7f36041049b0 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 0x7f36041026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f360410de10 0x7f3604103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 0x7f36041026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54212/0 (socket says 172.21.15.27:54212) 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 -- 172.21.15.27:0/3495511829 learned_addr learned my addr 172.21.15.27:0/3495511829 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3608a00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3604105300 0x7f3604102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 -- 172.21.15.27:0/3495511829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3604105300 msgr2=0x7f3604102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3604105300 0x7f3604102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 -- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f360410de10 msgr2=0x7f3604103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f360410de10 0x7f3604103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.950+0000 7f3609400640 1 -- 172.21.15.27:0/3495511829 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36041a8740 con 0x7f36041049b0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.951+0000 7f3609400640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 0x7f36041026f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005940 tx=0x7f35f80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.951+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f803f030 con 0x7f36041049b0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.951+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f80044c0 con 0x7f36041049b0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.951+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36041a89d0 con 0x7f36041049b0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.951+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8035070 con 0x7f36041049b0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.951+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041a8eb0 con 0x7f36041049b0 2024-09-21T01:21:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.952+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360406c2c0 con 0x7f36041049b0 2024-09-21T01:21:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.955+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f35f8042030 con 0x7f36041049b0 2024-09-21T01:21:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.957+0000 7f3602000640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f35ec0b65f0 0x7f35ec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.957+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f35f8107fb0 con 0x7f36041049b0 2024-09-21T01:21:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.957+0000 7f3608a00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f35ec0b65f0 0x7f35ec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:15.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.958+0000 7f3608a00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f35ec0b65f0 0x7f35ec0b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f360410fce0 tx=0x7f35f0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:15.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:15.959+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35f80d25d0 con 0x7f36041049b0 2024-09-21T01:21:16.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.156+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3604101d30 con 0x7f35ec0b65f0 2024-09-21T01:21:16.157 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:15 smithi027 ceph-mon[30375]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:16.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.162+0000 7f3602000640 1 -- 172.21.15.27:0/3495511829 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340418 (secure 0 0 0) 0x7f3604101d30 con 0x7f35ec0b65f0 2024-09-21T01:21:16.168 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:16.169 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:21:16.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f35ec0b65f0 msgr2=0x7f35ec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:16.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f35ec0b65f0 0x7f35ec0b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f360410fce0 tx=0x7f35f0005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:16.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 msgr2=0x7f36041026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 0x7f36041026f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005940 tx=0x7f35f80040c0 comp rx=0 tx=0).stop 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 shutdown_connections 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f35ec0b65f0 0x7f35ec0b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f360410de10 0x7f3604103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3604105300 0x7f3604102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 --2- 172.21.15.27:0/3495511829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36041049b0 0x7f36041026f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.167+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 >> 172.21.15.27:0/3495511829 conn(0x7f3604079760 msgr2=0x7f3604100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.168+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 shutdown_connections 2024-09-21T01:21:16.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:16.168+0000 7f360be00640 1 -- 172.21.15.27:0/3495511829 wait complete. 2024-09-21T01:21:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:15 smithi089 ceph-mon[34486]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:16.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:15 smithi098 ceph-mon[34372]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-21T01:21:14.454680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664018,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":739,"num_read_kb":498,"num_write":375,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3019356,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372092,"kb_used_data":14444,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569732,"statfs":{"total":1151940427776,"available":1151559405568,"internally_reserved":0,"allocated":14790656,"data_stored":8385164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004088"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":128,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:12.506544+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:21:12.506544+0000","last_peered":"2024-09-21T01:21:12.506544+0000","last_clean":"2024-09-21T01:21:12.506544+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:21:12.506544+0000","last_undegraded":"2024-09-21T01:21:12.506544+0000","last_fullsized":"2024-09-21T01:21:12.506544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166928+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:20:42.166928+0000","last_peered":"2024-09-21T01:20:42.166928+0000","last_clean":"2024-09-21T01:20:42.166928+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:20:42.166928+0000","last_undegraded":"2024-09-21T01:20:42.166928+0000","last_fullsized":"2024-09-21T01:20:42.166928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166049+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:20:42.166049+0000","last_peered":"2024-09-21T01:20:42.166049+0000","last_clean":"2024-09-21T01:20:42.166049+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:20:42.166049+0000","last_undegraded":"2024-09-21T01:20:42.166049+0000","last_fullsized":"2024-09-21T01:20:42.166049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213160+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:20:42.213160+0000","last_peered":"2024-09-21T01:20:42.213160+0000","last_clean":"2024-09-21T01:20:42.213160+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:20:42.213160+0000","last_undegraded":"2024-09-21T01:20:42.213160+0000","last_fullsized":"2024-09-21T01:20:42.213160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171832+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:20:42.171832+0000","last_peered":"2024-09-21T01:20:42.171832+0000","last_clean":"2024-09-21T01:20:42.171832+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:20:42.171832+0000","last_undegraded":"2024-09-21T01:20:42.171832+0000","last_fullsized":"2024-09-21T01:20:42.171832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168709+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:20:42.168709+0000","last_peered":"2024-09-21T01:20:42.168709+0000","last_clean":"2024-09-21T01:20:42.168709+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:20:42.168709+0000","last_undegraded":"2024-09-21T01:20:42.168709+0000","last_fullsized":"2024-09-21T01:20:42.168709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172079+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:20:42.172079+0000","last_peered":"2024-09-21T01:20:42.172079+0000","last_clean":"2024-09-21T01:20:42.172079+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:20:42.172079+0000","last_undegraded":"2024-09-21T01:20:42.172079+0000","last_fullsized":"2024-09-21T01:20:42.172079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173133+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:20:42.173133+0000","last_peered":"2024-09-21T01:20:42.173133+0000","last_clean":"2024-09-21T01:20:42.173133+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:20:42.173133+0000","last_undegraded":"2024-09-21T01:20:42.173133+0000","last_fullsized":"2024-09-21T01:20:42.173133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169342+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:20:42.169342+0000","last_peered":"2024-09-21T01:20:42.169342+0000","last_clean":"2024-09-21T01:20:42.169342+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:20:42.169342+0000","last_undegraded":"2024-09-21T01:20:42.169342+0000","last_fullsized":"2024-09-21T01:20:42.169342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172395+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:42.172395+0000","last_peered":"2024-09-21T01:20:42.172395+0000","last_clean":"2024-09-21T01:20:42.172395+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:42.172395+0000","last_undegraded":"2024-09-21T01:20:42.172395+0000","last_fullsized":"2024-09-21T01:20:42.172395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174059+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:20:42.174059+0000","last_peered":"2024-09-21T01:20:42.174059+0000","last_clean":"2024-09-21T01:20:42.174059+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:20:42.174059+0000","last_undegraded":"2024-09-21T01:20:42.174059+0000","last_fullsized":"2024-09-21T01:20:42.174059+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169252+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:20:42.169252+0000","last_peered":"2024-09-21T01:20:42.169252+0000","last_clean":"2024-09-21T01:20:42.169252+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:20:42.169252+0000","last_undegraded":"2024-09-21T01:20:42.169252+0000","last_fullsized":"2024-09-21T01:20:42.169252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166612+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:20:42.166612+0000","last_peered":"2024-09-21T01:20:42.166612+0000","last_clean":"2024-09-21T01:20:42.166612+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:20:42.166612+0000","last_undegraded":"2024-09-21T01:20:42.166612+0000","last_fullsized":"2024-09-21T01:20:42.166612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171717+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:42.171717+0000","last_peered":"2024-09-21T01:20:42.171717+0000","last_clean":"2024-09-21T01:20:42.171717+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:42.171717+0000","last_undegraded":"2024-09-21T01:20:42.171717+0000","last_fullsized":"2024-09-21T01:20:42.171717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170083+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:20:42.170083+0000","last_peered":"2024-09-21T01:20:42.170083+0000","last_clean":"2024-09-21T01:20:42.170083+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:20:42.170083+0000","last_undegraded":"2024-09-21T01:20:42.170083+0000","last_fullsized":"2024-09-21T01:20:42.170083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212732+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:20:42.212732+0000","last_peered":"2024-09-21T01:20:42.212732+0000","last_clean":"2024-09-21T01:20:42.212732+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:20:42.212732+0000","last_undegraded":"2024-09-21T01:20:42.212732+0000","last_fullsized":"2024-09-21T01:20:42.212732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166193+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:42.166193+0000","last_peered":"2024-09-21T01:20:42.166193+0000","last_clean":"2024-09-21T01:20:42.166193+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:42.166193+0000","last_undegraded":"2024-09-21T01:20:42.166193+0000","last_fullsized":"2024-09-21T01:20:42.166193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169923+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:20:42.169923+0000","last_peered":"2024-09-21T01:20:42.169923+0000","last_clean":"2024-09-21T01:20:42.169923+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:20:42.169923+0000","last_undegraded":"2024-09-21T01:20:42.169923+0000","last_fullsized":"2024-09-21T01:20:42.169923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168722+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:20:42.168722+0000","last_peered":"2024-09-21T01:20:42.168722+0000","last_clean":"2024-09-21T01:20:42.168722+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:20:42.168722+0000","last_undegraded":"2024-09-21T01:20:42.168722+0000","last_fullsized":"2024-09-21T01:20:42.168722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212760+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:42.212760+0000","last_peered":"2024-09-21T01:20:42.212760+0000","last_clean":"2024-09-21T01:20:42.212760+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:42.212760+0000","last_undegraded":"2024-09-21T01:20:42.212760+0000","last_fullsized":"2024-09-21T01:20:42.212760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166926+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:20:42.166926+0000","last_peered":"2024-09-21T01:20:42.166926+0000","last_clean":"2024-09-21T01:20:42.166926+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:20:42.166926+0000","last_undegraded":"2024-09-21T01:20:42.166926+0000","last_fullsized":"2024-09-21T01:20:42.166926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171243+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:20:42.171243+0000","last_peered":"2024-09-21T01:20:42.171243+0000","last_clean":"2024-09-21T01:20:42.171243+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:20:42.171243+0000","last_undegraded":"2024-09-21T01:20:42.171243+0000","last_fullsized":"2024-09-21T01:20:42.171243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173345+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:42.173345+0000","last_peered":"2024-09-21T01:20:42.173345+0000","last_clean":"2024-09-21T01:20:42.173345+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:42.173345+0000","last_undegraded":"2024-09-21T01:20:42.173345+0000","last_fullsized":"2024-09-21T01:20:42.173345+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170064+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:20:42.170064+0000","last_peered":"2024-09-21T01:20:42.170064+0000","last_clean":"2024-09-21T01:20:42.170064+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.170064+0000","last_undegraded":"2024-09-21T01:20:42.170064+0000","last_fullsized":"2024-09-21T01:20:42.170064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171766+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:20:42.171766+0000","last_peered":"2024-09-21T01:20:42.171766+0000","last_clean":"2024-09-21T01:20:42.171766+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:20:42.171766+0000","last_undegraded":"2024-09-21T01:20:42.171766+0000","last_fullsized":"2024-09-21T01:20:42.171766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170377+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:42.170377+0000","last_peered":"2024-09-21T01:20:42.170377+0000","last_clean":"2024-09-21T01:20:42.170377+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:42.170377+0000","last_undegraded":"2024-09-21T01:20:42.170377+0000","last_fullsized":"2024-09-21T01:20:42.170377+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166231+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:20:42.166231+0000","last_peered":"2024-09-21T01:20:42.166231+0000","last_clean":"2024-09-21T01:20:42.166231+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:20:42.166231+0000","last_undegraded":"2024-09-21T01:20:42.166231+0000","last_fullsized":"2024-09-21T01:20:42.166231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213147+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:20:42.213147+0000","last_peered":"2024-09-21T01:20:42.213147+0000","last_clean":"2024-09-21T01:20:42.213147+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:20:42.213147+0000","last_undegraded":"2024-09-21T01:20:42.213147+0000","last_fullsized":"2024-09-21T01:20:42.213147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'16","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167427+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:20:42.167427+0000","last_peered":"2024-09-21T01:20:42.167427+0000","last_clean":"2024-09-21T01:20:42.167427+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:20:42.167427+0000","last_undegraded":"2024-09-21T01:20:42.167427+0000","last_fullsized":"2024-09-21T01:20:42.167427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212422+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:20:42.212422+0000","last_peered":"2024-09-21T01:20:42.212422+0000","last_clean":"2024-09-21T01:20:42.212422+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.212422+0000","last_undegraded":"2024-09-21T01:20:42.212422+0000","last_fullsized":"2024-09-21T01:20:42.212422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368647+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:20:42.368647+0000","last_peered":"2024-09-21T01:20:42.368647+0000","last_clean":"2024-09-21T01:20:42.368647+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:20:42.368647+0000","last_undegraded":"2024-09-21T01:20:42.368647+0000","last_fullsized":"2024-09-21T01:20:42.368647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.185170+0000","last_change":"2024-09-21T01:20:42.184446+0000","last_active":"2024-09-21T01:20:42.185170+0000","last_peered":"2024-09-21T01:20:42.185170+0000","last_clean":"2024-09-21T01:20:42.185170+0000","last_became_active":"2024-09-21T01:20:42.172338+0000","last_became_peered":"2024-09-21T01:20:42.172338+0000","last_unstale":"2024-09-21T01:20:42.185170+0000","last_undegraded":"2024-09-21T01:20:42.185170+0000","last_fullsized":"2024-09-21T01:20:42.185170+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:12.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168686+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:20:42.168686+0000","last_peered":"2024-09-21T01:20:42.168686+0000","last_clean":"2024-09-21T01:20:42.168686+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:20:42.168686+0000","last_undegraded":"2024-09-21T01:20:42.168686+0000","last_fullsized":"2024-09-21T01:20:42.168686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169233+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:20:42.169233+0000","last_peered":"2024-09-21T01:20:42.169233+0000","last_clean":"2024-09-21T01:20:42.169233+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:20:42.169233+0000","last_undegraded":"2024-09-21T01:20:42.169233+0000","last_fullsized":"2024-09-21T01:20:42.169233+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167193+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:20:42.167193+0000","last_peered":"2024-09-21T01:20:42.167193+0000","last_clean":"2024-09-21T01:20:42.167193+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:20:42.167193+0000","last_undegraded":"2024-09-21T01:20:42.167193+0000","last_fullsized":"2024-09-21T01:20:42.167193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171149+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:20:42.171149+0000","last_peered":"2024-09-21T01:20:42.171149+0000","last_clean":"2024-09-21T01:20:42.171149+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:20:42.171149+0000","last_undegraded":"2024-09-21T01:20:42.171149+0000","last_fullsized":"2024-09-21T01:20:42.171149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168900+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:20:42.168900+0000","last_peered":"2024-09-21T01:20:42.168900+0000","last_clean":"2024-09-21T01:20:42.168900+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:20:42.168900+0000","last_undegraded":"2024-09-21T01:20:42.168900+0000","last_fullsized":"2024-09-21T01:20:42.168900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:09.502229+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:21:09.502229+0000","last_peered":"2024-09-21T01:21:09.502229+0000","last_clean":"2024-09-21T01:21:09.502229+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:21:09.502229+0000","last_undegraded":"2024-09-21T01:21:09.502229+0000","last_fullsized":"2024-09-21T01:21:09.502229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368463+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:20:42.368463+0000","last_peered":"2024-09-21T01:20:42.368463+0000","last_clean":"2024-09-21T01:20:42.368463+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:20:42.368463+0000","last_undegraded":"2024-09-21T01:20:42.368463+0000","last_fullsized":"2024-09-21T01:20:42.368463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173332+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:20:42.173332+0000","last_peered":"2024-09-21T01:20:42.173332+0000","last_clean":"2024-09-21T01:20:42.173332+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:20:42.173332+0000","last_undegraded":"2024-09-21T01:20:42.173332+0000","last_fullsized":"2024-09-21T01:20:42.173332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:12.510111+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:21:12.510111+0000","last_peered":"2024-09-21T01:21:12.510111+0000","last_clean":"2024-09-21T01:21:12.510111+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:21:12.510111+0000","last_undegraded":"2024-09-21T01:21:12.510111+0000","last_fullsized":"2024-09-21T01:21:12.510111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368606+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:20:42.368606+0000","last_peered":"2024-09-21T01:20:42.368606+0000","last_clean":"2024-09-21T01:20:42.368606+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:20:42.368606+0000","last_undegraded":"2024-09-21T01:20:42.368606+0000","last_fullsized":"2024-09-21T01:20:42.368606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368178+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:20:42.368178+0000","last_peered":"2024-09-21T01:20:42.368178+0000","last_clean":"2024-09-21T01:20:42.368178+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:20:42.368178+0000","last_undegraded":"2024-09-21T01:20:42.368178+0000","last_fullsized":"2024-09-21T01:20:42.368178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166486+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:42.166486+0000","last_peered":"2024-09-21T01:20:42.166486+0000","last_clean":"2024-09-21T01:20:42.166486+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:42.166486+0000","last_undegraded":"2024-09-21T01:20:42.166486+0000","last_fullsized":"2024-09-21T01:20:42.166486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171822+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:20:42.171822+0000","last_peered":"2024-09-21T01:20:42.171822+0000","last_clean":"2024-09-21T01:20:42.171822+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:20:42.171822+0000","last_undegraded":"2024-09-21T01:20:42.171822+0000","last_fullsized":"2024-09-21T01:20:42.171822+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166644+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:20:42.166644+0000","last_peered":"2024-09-21T01:20:42.166644+0000","last_clean":"2024-09-21T01:20:42.166644+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:20:42.166644+0000","last_undegraded":"2024-09-21T01:20:42.166644+0000","last_fullsized":"2024-09-21T01:20:42.166644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169791+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:42.169791+0000","last_peered":"2024-09-21T01:20:42.169791+0000","last_clean":"2024-09-21T01:20:42.169791+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:42.169791+0000","last_undegraded":"2024-09-21T01:20:42.169791+0000","last_fullsized":"2024-09-21T01:20:42.169791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166597+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:20:42.166597+0000","last_peered":"2024-09-21T01:20:42.166597+0000","last_clean":"2024-09-21T01:20:42.166597+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:20:42.166597+0000","last_undegraded":"2024-09-21T01:20:42.166597+0000","last_fullsized":"2024-09-21T01:20:42.166597+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166585+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:20:42.166585+0000","last_peered":"2024-09-21T01:20:42.166585+0000","last_clean":"2024-09-21T01:20:42.166585+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:20:42.166585+0000","last_undegraded":"2024-09-21T01:20:42.166585+0000","last_fullsized":"2024-09-21T01:20:42.166585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166859+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:20:42.166859+0000","last_peered":"2024-09-21T01:20:42.166859+0000","last_clean":"2024-09-21T01:20:42.166859+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:20:42.166859+0000","last_undegraded":"2024-09-21T01:20:42.166859+0000","last_fullsized":"2024-09-21T01:20:42.166859+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:08.226573+0000","last_change":"2024-09-21T01:20:42.188731+0000","last_active":"2024-09-21T01:21:08.226573+0000","last_peered":"2024-09-21T01:21:08.226573+0000","last_clean":"2024-09-21T01:21:08.226573+0000","last_became_active":"2024-09-21T01:20:42.172041+0000","last_became_peered":"2024-09-21T01:20:42.172041+0000","last_unstale":"2024-09-21T01:21:08.226573+0000","last_undegraded":"2024-09-21T01:21:08.226573+0000","last_fullsized":"2024-09-21T01:21:08.226573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:42.392074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173015+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:20:42.173015+0000","last_peered":"2024-09-21T01:20:42.173015+0000","last_clean":"2024-09-21T01:20:42.173015+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:20:42.173015+0000","last_undegraded":"2024-09-21T01:20:42.173015+0000","last_fullsized":"2024-09-21T01:20:42.173015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166727+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:20:42.166727+0000","last_peered":"2024-09-21T01:20:42.166727+0000","last_clean":"2024-09-21T01:20:42.166727+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:20:42.166727+0000","last_undegraded":"2024-09-21T01:20:42.166727+0000","last_fullsized":"2024-09-21T01:20:42.166727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174353+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:20:42.174353+0000","last_peered":"2024-09-21T01:20:42.174353+0000","last_clean":"2024-09-21T01:20:42.174353+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:20:42.174353+0000","last_undegraded":"2024-09-21T01:20:42.174353+0000","last_fullsized":"2024-09-21T01:20:42.174353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171285+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:20:42.171285+0000","last_peered":"2024-09-21T01:20:42.171285+0000","last_clean":"2024-09-21T01:20:42.171285+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:20:42.171285+0000","last_undegraded":"2024-09-21T01:20:42.171285+0000","last_fullsized":"2024-09-21T01:20:42.171285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167009+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:20:42.167009+0000","last_peered":"2024-09-21T01:20:42.167009+0000","last_clean":"2024-09-21T01:20:42.167009+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:20:42.167009+0000","last_undegraded":"2024-09-21T01:20:42.167009+0000","last_fullsized":"2024-09-21T01:20:42.167009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368627+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:42.368627+0000","last_peered":"2024-09-21T01:20:42.368627+0000","last_clean":"2024-09-21T01:20:42.368627+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:42.368627+0000","last_undegraded":"2024-09-21T01:20:42.368627+0000","last_fullsized":"2024-09-21T01:20:42.368627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173098+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:20:42.173098+0000","last_peered":"2024-09-21T01:20:42.173098+0000","last_clean":"2024-09-21T01:20:42.173098+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:20:42.173098+0000","last_undegraded":"2024-09-21T01:20:42.173098+0000","last_fullsized":"2024-09-21T01:20:42.173098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169084+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:20:42.169084+0000","last_peered":"2024-09-21T01:20:42.169084+0000","last_clean":"2024-09-21T01:20:42.169084+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:20:42.169084+0000","last_undegraded":"2024-09-21T01:20:42.169084+0000","last_fullsized":"2024-09-21T01:20:42.169084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.189333+0000","last_change":"2024-09-21T01:20:42.188808+0000","last_active":"2024-09-21T01:20:42.189333+0000","last_peered":"2024-09-21T01:20:42.189333+0000","last_clean":"2024-09-21T01:20:42.189333+0000","last_became_active":"2024-09-21T01:20:42.175348+0000","last_became_peered":"2024-09-21T01:20:42.175348+0000","last_unstale":"2024-09-21T01:20:42.189333+0000","last_undegraded":"2024-09-21T01:20:42.189333+0000","last_fullsized":"2024-09-21T01:20:42.189333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.746569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173522+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:20:42.173522+0000","last_peered":"2024-09-21T01:20:42.173522+0000","last_clean":"2024-09-21T01:20:42.173522+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:20:42.173522+0000","last_undegraded":"2024-09-21T01:20:42.173522+0000","last_fullsized":"2024-09-21T01:20:42.173522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173643+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:20:42.173643+0000","last_peered":"2024-09-21T01:20:42.173643+0000","last_clean":"2024-09-21T01:20:42.173643+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:20:42.173643+0000","last_undegraded":"2024-09-21T01:20:42.173643+0000","last_fullsized":"2024-09-21T01:20:42.173643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171162+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:42.171162+0000","last_peered":"2024-09-21T01:20:42.171162+0000","last_clean":"2024-09-21T01:20:42.171162+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:42.171162+0000","last_undegraded":"2024-09-21T01:20:42.171162+0000","last_fullsized":"2024-09-21T01:20:42.171162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368583+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:20:42.368583+0000","last_peered":"2024-09-21T01:20:42.368583+0000","last_clean":"2024-09-21T01:20:42.368583+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:20:42.368583+0000","last_undegraded":"2024-09-21T01:20:42.368583+0000","last_fullsized":"2024-09-21T01:20:42.368583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166303+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:20:42.166303+0000","last_peered":"2024-09-21T01:20:42.166303+0000","last_clean":"2024-09-21T01:20:42.166303+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:20:42.166303+0000","last_undegraded":"2024-09-21T01:20:42.166303+0000","last_fullsized":"2024-09-21T01:20:42.166303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:08.504880+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:21:08.504880+0000","last_peered":"2024-09-21T01:21:08.504880+0000","last_clean":"2024-09-21T01:21:08.504880+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:21:08.504880+0000","last_undegraded":"2024-09-21T01:21:08.504880+0000","last_fullsized":"2024-09-21T01:21:08.504880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172538+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:20:42.172538+0000","last_peered":"2024-09-21T01:20:42.172538+0000","last_clean":"2024-09-21T01:20:42.172538+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:20:42.172538+0000","last_undegraded":"2024-09-21T01:20:42.172538+0000","last_fullsized":"2024-09-21T01:20:42.172538+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166615+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:20:42.166615+0000","last_peered":"2024-09-21T01:20:42.166615+0000","last_clean":"2024-09-21T01:20:42.166615+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:20:42.166615+0000","last_undegraded":"2024-09-21T01:20:42.166615+0000","last_fullsized":"2024-09-21T01:20:42.166615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173908+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:42.173908+0000","last_peered":"2024-09-21T01:20:42.173908+0000","last_clean":"2024-09-21T01:20:42.173908+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:42.173908+0000","last_undegraded":"2024-09-21T01:20:42.173908+0000","last_fullsized":"2024-09-21T01:20:42.173908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168692+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:20:42.168692+0000","last_peered":"2024-09-21T01:20:42.168692+0000","last_clean":"2024-09-21T01:20:42.168692+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:20:42.168692+0000","last_undegraded":"2024-09-21T01:20:42.168692+0000","last_fullsized":"2024-09-21T01:20:42.168692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368243+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:20:42.368243+0000","last_peered":"2024-09-21T01:20:42.368243+0000","last_clean":"2024-09-21T01:20:42.368243+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:20:42.368243+0000","last_undegraded":"2024-09-21T01:20:42.368243+0000","last_fullsized":"2024-09-21T01:20:42.368243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166554+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:42.166554+0000","last_peered":"2024-09-21T01:20:42.166554+0000","last_clean":"2024-09-21T01:20:42.166554+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:42.166554+0000","last_undegraded":"2024-09-21T01:20:42.166554+0000","last_fullsized":"2024-09-21T01:20:42.166554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173913+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:20:42.173913+0000","last_peered":"2024-09-21T01:20:42.173913+0000","last_clean":"2024-09-21T01:20:42.173913+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:20:42.173913+0000","last_undegraded":"2024-09-21T01:20:42.173913+0000","last_fullsized":"2024-09-21T01:20:42.173913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171904+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:20:42.171904+0000","last_peered":"2024-09-21T01:20:42.171904+0000","last_clean":"2024-09-21T01:20:42.171904+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:20:42.171904+0000","last_undegraded":"2024-09-21T01:20:42.171904+0000","last_fullsized":"2024-09-21T01:20:42.171904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:10.218958+0000","last_change":"2024-09-21T01:20:42.182111+0000","last_active":"2024-09-21T01:21:10.218958+0000","last_peered":"2024-09-21T01:21:10.218958+0000","last_clean":"2024-09-21T01:21:10.218958+0000","last_became_active":"2024-09-21T01:20:42.174484+0000","last_became_peered":"2024-09-21T01:20:42.174484+0000","last_unstale":"2024-09-21T01:21:10.218958+0000","last_undegraded":"2024-09-21T01:21:10.218958+0000","last_fullsized":"2024-09-21T01:21:10.218958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:45.448208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66212,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":133,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":66212,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166800+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:20:42.166800+0000","last_peered":"2024-09-21T01:20:42.166800+0000","last_clean":"2024-09-21T01:20:42.166800+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:20:42.166800+0000","last_undegraded":"2024-09-21T01:20:42.166800+0000","last_fullsized":"2024-09-21T01:20:42.166800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173328+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:20:42.173328+0000","last_peered":"2024-09-21T01:20:42.173328+0000","last_clean":"2024-09-21T01:20:42.173328+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:20:42.173328+0000","last_undegraded":"2024-09-21T01:20:42.173328+0000","last_fullsized":"2024-09-21T01:20:42.173328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174235+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:42.174235+0000","last_peered":"2024-09-21T01:20:42.174235+0000","last_clean":"2024-09-21T01:20:42.174235+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:42.174235+0000","last_undegraded":"2024-09-21T01:20:42.174235+0000","last_fullsized":"2024-09-21T01:20:42.174235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169546+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:20:42.169546+0000","last_peered":"2024-09-21T01:20:42.169546+0000","last_clean":"2024-09-21T01:20:42.169546+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:20:42.169546+0000","last_undegraded":"2024-09-21T01:20:42.169546+0000","last_fullsized":"2024-09-21T01:20:42.169546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171822+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:20:42.171822+0000","last_peered":"2024-09-21T01:20:42.171822+0000","last_clean":"2024-09-21T01:20:42.171822+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:20:42.171822+0000","last_undegraded":"2024-09-21T01:20:42.171822+0000","last_fullsized":"2024-09-21T01:20:42.171822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167088+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:42.167088+0000","last_peered":"2024-09-21T01:20:42.167088+0000","last_clean":"2024-09-21T01:20:42.167088+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:42.167088+0000","last_undegraded":"2024-09-21T01:20:42.167088+0000","last_fullsized":"2024-09-21T01:20:42.167088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166526+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:20:42.166526+0000","last_peered":"2024-09-21T01:20:42.166526+0000","last_clean":"2024-09-21T01:20:42.166526+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:20:42.166526+0000","last_undegraded":"2024-09-21T01:20:42.166526+0000","last_fullsized":"2024-09-21T01:20:42.166526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173219+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:20:42.173219+0000","last_peered":"2024-09-21T01:20:42.173219+0000","last_clean":"2024-09-21T01:20:42.173219+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:20:42.173219+0000","last_undegraded":"2024-09-21T01:20:42.173219+0000","last_fullsized":"2024-09-21T01:20:42.173219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":179,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:09.502242+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:21:09.502242+0000","last_peered":"2024-09-21T01:21:09.502242+0000","last_clean":"2024-09-21T01:21:09.502242+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:21:09.502242+0000","last_undegraded":"2024-09-21T01:21:09.502242+0000","last_fullsized":"2024-09-21T01:21:09.502242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212527+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:20:42.212527+0000","last_peered":"2024-09-21T01:20:42.212527+0000","last_clean":"2024-09-21T01:20:42.212527+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:20:42.212527+0000","last_undegraded":"2024-09-21T01:20:42.212527+0000","last_fullsized":"2024-09-21T01:20:42.212527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171628+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:20:42.171628+0000","last_peered":"2024-09-21T01:20:42.171628+0000","last_clean":"2024-09-21T01:20:42.171628+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:20:42.171628+0000","last_undegraded":"2024-09-21T01:20:42.171628+0000","last_fullsized":"2024-09-21T01:20:42.171628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166658+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:20:42.166658+0000","last_peered":"2024-09-21T01:20:42.166658+0000","last_clean":"2024-09-21T01:20:42.166658+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:20:42.166658+0000","last_undegraded":"2024-09-21T01:20:42.166658+0000","last_fullsized":"2024-09-21T01:20:42.166658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173384+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:42.173384+0000","last_peered":"2024-09-21T01:20:42.173384+0000","last_clean":"2024-09-21T01:20:42.173384+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:42.173384+0000","last_undegraded":"2024-09-21T01:20:42.173384+0000","last_fullsized":"2024-09-21T01:20:42.173384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169092+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:20:42.169092+0000","last_peered":"2024-09-21T01:20:42.169092+0000","last_clean":"2024-09-21T01:20:42.169092+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:20:42.169092+0000","last_undegraded":"2024-09-21T01:20:42.169092+0000","last_fullsized":"2024-09-21T01:20:42.169092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212586+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:20:42.212586+0000","last_peered":"2024-09-21T01:20:42.212586+0000","last_clean":"2024-09-21T01:20:42.212586+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:20:42.212586+0000","last_undegraded":"2024-09-21T01:20:42.212586+0000","last_fullsized":"2024-09-21T01:20:42.212586+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368099+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:42.368099+0000","last_peered":"2024-09-21T01:20:42.368099+0000","last_clean":"2024-09-21T01:20:42.368099+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:42.368099+0000","last_undegraded":"2024-09-21T01:20:42.368099+0000","last_fullsized":"2024-09-21T01:20:42.368099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169896+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:42.169896+0000","last_peered":"2024-09-21T01:20:42.169896+0000","last_clean":"2024-09-21T01:20:42.169896+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:42.169896+0000","last_undegraded":"2024-09-21T01:20:42.169896+0000","last_fullsized":"2024-09-21T01:20:42.169896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173888+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:20:42.173888+0000","last_peered":"2024-09-21T01:20:42.173888+0000","last_clean":"2024-09-21T01:20:42.173888+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:20:42.173888+0000","last_undegraded":"2024-09-21T01:20:42.173888+0000","last_fullsized":"2024-09-21T01:20:42.173888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166218+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:20:42.166218+0000","last_peered":"2024-09-21T01:20:42.166218+0000","last_clean":"2024-09-21T01:20:42.166218+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:20:42.166218+0000","last_undegraded":"2024-09-21T01:20:42.166218+0000","last_fullsized":"2024-09-21T01:20:42.166218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:10.504491+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:21:10.504491+0000","last_peered":"2024-09-21T01:21:10.504491+0000","last_clean":"2024-09-21T01:21:10.504491+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:21:10.504491+0000","last_undegraded":"2024-09-21T01:21:10.504491+0000","last_fullsized":"2024-09-21T01:21:10.504491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171380+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:20:42.171380+0000","last_peered":"2024-09-21T01:20:42.171380+0000","last_clean":"2024-09-21T01:20:42.171380+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:20:42.171380+0000","last_undegraded":"2024-09-21T01:20:42.171380+0000","last_fullsized":"2024-09-21T01:20:42.171380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166508+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:20:42.166508+0000","last_peered":"2024-09-21T01:20:42.166508+0000","last_clean":"2024-09-21T01:20:42.166508+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:20:42.166508+0000","last_undegraded":"2024-09-21T01:20:42.166508+0000","last_fullsized":"2024-09-21T01:20:42.166508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168775+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:20:42.168775+0000","last_peered":"2024-09-21T01:20:42.168775+0000","last_clean":"2024-09-21T01:20:42.168775+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:20:42.168775+0000","last_undegraded":"2024-09-21T01:20:42.168775+0000","last_fullsized":"2024-09-21T01:20:42.168775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166554+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:20:42.166554+0000","last_peered":"2024-09-21T01:20:42.166554+0000","last_clean":"2024-09-21T01:20:42.166554+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:20:42.166554+0000","last_undegraded":"2024-09-21T01:20:42.166554+0000","last_fullsized":"2024-09-21T01:20:42.166554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166606+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:20:42.166606+0000","last_peered":"2024-09-21T01:20:42.166606+0000","last_clean":"2024-09-21T01:20:42.166606+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:20:42.166606+0000","last_undegraded":"2024-09-21T01:20:42.166606+0000","last_fullsized":"2024-09-21T01:20:42.166606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171403+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:20:42.171403+0000","last_peered":"2024-09-21T01:20:42.171403+0000","last_clean":"2024-09-21T01:20:42.171403+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:20:42.171403+0000","last_undegraded":"2024-09-21T01:20:42.171403+0000","last_fullsized":"2024-09-21T01:20:42.171403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368601+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:20:42.368601+0000","last_peered":"2024-09-21T01:20:42.368601+0000","last_clean":"2024-09-21T01:20:42.368601+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:20:42.368601+0000","last_undegraded":"2024-09-21T01:20:42.368601+0000","last_fullsized":"2024-09-21T01:20:42.368601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169605+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:20:42.169605+0000","last_peered":"2024-09-21T01:20:42.169605+0000","last_clean":"2024-09-21T01:20:42.169605+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:20:42.169605+0000","last_undegraded":"2024-09-21T01:20:42.169605+0000","last_fullsized":"2024-09-21T01:20:42.169605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166579+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:20:42.166579+0000","last_peered":"2024-09-21T01:20:42.166579+0000","last_clean":"2024-09-21T01:20:42.166579+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:20:42.166579+0000","last_undegraded":"2024-09-21T01:20:42.166579+0000","last_fullsized":"2024-09-21T01:20:42.166579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213117+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:20:42.213117+0000","last_peered":"2024-09-21T01:20:42.213117+0000","last_clean":"2024-09-21T01:20:42.213117+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:20:42.213117+0000","last_undegraded":"2024-09-21T01:20:42.213117+0000","last_fullsized":"2024-09-21T01:20:42.213117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173192+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:20:42.173192+0000","last_peered":"2024-09-21T01:20:42.173192+0000","last_clean":"2024-09-21T01:20:42.173192+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:20:42.173192+0000","last_undegraded":"2024-09-21T01:20:42.173192+0000","last_fullsized":"2024-09-21T01:20:42.173192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212633+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:20:42.212633+0000","last_peered":"2024-09-21T01:20:42.212633+0000","last_clean":"2024-09-21T01:20:42.212633+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:20:42.212633+0000","last_undegraded":"2024-09-21T01:20:42.212633+0000","last_fullsized":"2024-09-21T01:20:42.212633+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173219+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:20:42.173219+0000","last_peered":"2024-09-21T01:20:42.173219+0000","last_clean":"2024-09-21T01:20:42.173219+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:20:42.173219+0000","last_undegraded":"2024-09-21T01:20:42.173219+0000","last_fullsized":"2024-09-21T01:20:42.173219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368426+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:20:42.368426+0000","last_peered":"2024-09-21T01:20:42.368426+0000","last_clean":"2024-09-21T01:20:42.368426+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:20:42.368426+0000","last_undegraded":"2024-09-21T01:20:42.368426+0000","last_fullsized":"2024-09-21T01:20:42.368426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168697+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:20:42.168697+0000","last_peered":"2024-09-21T01:20:42.168697+0000","last_clean":"2024-09-21T01:20:42.168697+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:20:42.168697+0000","last_undegraded":"2024-09-21T01:20:42.168697+0000","last_fullsized":"2024-09-21T01:20:42.168697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166639+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:20:42.166639+0000","last_peered":"2024-09-21T01:20:42.166639+0000","last_clean":"2024-09-21T01:20:42.166639+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:20:42.166639+0000","last_undegraded":"2024-09-21T01:20:42.166639+0000","last_fullsized":"2024-09-21T01:20:42.166639+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167341+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:20:42.167341+0000","last_peered":"2024-09-21T01:20:42.167341+0000","last_clean":"2024-09-21T01:20:42.167341+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:20:42.167341+0000","last_undegraded":"2024-09-21T01:20:42.167341+0000","last_fullsized":"2024-09-21T01:20:42.167341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169662+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:20:42.169662+0000","last_peered":"2024-09-21T01:20:42.169662+0000","last_clean":"2024-09-21T01:20:42.169662+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:20:42.169662+0000","last_undegraded":"2024-09-21T01:20:42.169662+0000","last_fullsized":"2024-09-21T01:20:42.169662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172681+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:20:42.172681+0000","last_peered":"2024-09-21T01:20:42.172681+0000","last_clean":"2024-09-21T01:20:42.172681+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:20:42.172681+0000","last_undegraded":"2024-09-21T01:20:42.172681+0000","last_fullsized":"2024-09-21T01:20:42.172681+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171018+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:20:42.171018+0000","last_peered":"2024-09-21T01:20:42.171018+0000","last_clean":"2024-09-21T01:20:42.171018+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:20:42.171018+0000","last_undegraded":"2024-09-21T01:20:42.171018+0000","last_fullsized":"2024-09-21T01:20:42.171018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171106+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:20:42.171106+0000","last_peered":"2024-09-21T01:20:42.171106+0000","last_clean":"2024-09-21T01:20:42.171106+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:20:42.171106+0000","last_undegraded":"2024-09-21T01:20:42.171106+0000","last_fullsized":"2024-09-21T01:20:42.171106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368338+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:20:42.368338+0000","last_peered":"2024-09-21T01:20:42.368338+0000","last_clean":"2024-09-21T01:20:42.368338+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:20:42.368338+0000","last_undegraded":"2024-09-21T01:20:42.368338+0000","last_fullsized":"2024-09-21T01:20:42.368338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170161+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:20:42.170161+0000","last_peered":"2024-09-21T01:20:42.170161+0000","last_clean":"2024-09-21T01:20:42.170161+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:20:42.170161+0000","last_undegraded":"2024-09-21T01:20:42.170161+0000","last_fullsized":"2024-09-21T01:20:42.170161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166766+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:20:42.166766+0000","last_peered":"2024-09-21T01:20:42.166766+0000","last_clean":"2024-09-21T01:20:42.166766+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:20:42.166766+0000","last_undegraded":"2024-09-21T01:20:42.166766+0000","last_fullsized":"2024-09-21T01:20:42.166766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170044+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:20:42.170044+0000","last_peered":"2024-09-21T01:20:42.170044+0000","last_clean":"2024-09-21T01:20:42.170044+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:20:42.170044+0000","last_undegraded":"2024-09-21T01:20:42.170044+0000","last_fullsized":"2024-09-21T01:20:42.170044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173327+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:20:42.173327+0000","last_peered":"2024-09-21T01:20:42.173327+0000","last_clean":"2024-09-21T01:20:42.173327+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:20:42.173327+0000","last_undegraded":"2024-09-21T01:20:42.173327+0000","last_fullsized":"2024-09-21T01:20:42.173327+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212552+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:20:42.212552+0000","last_peered":"2024-09-21T01:20:42.212552+0000","last_clean":"2024-09-21T01:20:42.212552+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:20:42.212552+0000","last_undegraded":"2024-09-21T01:20:42.212552+0000","last_fullsized":"2024-09-21T01:20:42.212552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166340+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:20:42.166340+0000","last_peered":"2024-09-21T01:20:42.166340+0000","last_clean":"2024-09-21T01:20:42.166340+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:20:42.166340+0000","last_undegraded":"2024-09-21T01:20:42.166340+0000","last_fullsized":"2024-09-21T01:20:42.166340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169730+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:20:42.169730+0000","last_peered":"2024-09-21T01:20:42.169730+0000","last_clean":"2024-09-21T01:20:42.169730+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:20:42.169730+0000","last_undegraded":"2024-09-21T01:20:42.169730+0000","last_fullsized":"2024-09-21T01:20:42.169730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171574+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:20:42.171574+0000","last_peered":"2024-09-21T01:20:42.171574+0000","last_clean":"2024-09-21T01:20:42.171574+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:20:42.171574+0000","last_undegraded":"2024-09-21T01:20:42.171574+0000","last_fullsized":"2024-09-21T01:20:42.171574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170266+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:20:42.170266+0000","last_peered":"2024-09-21T01:20:42.170266+0000","last_clean":"2024-09-21T01:20:42.170266+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:20:42.170266+0000","last_undegraded":"2024-09-21T01:20:42.170266+0000","last_fullsized":"2024-09-21T01:20:42.170266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171568+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:20:42.171568+0000","last_peered":"2024-09-21T01:20:42.171568+0000","last_clean":"2024-09-21T01:20:42.171568+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:20:42.171568+0000","last_undegraded":"2024-09-21T01:20:42.171568+0000","last_fullsized":"2024-09-21T01:20:42.171568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171626+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:20:42.171626+0000","last_peered":"2024-09-21T01:20:42.171626+0000","last_clean":"2024-09-21T01:20:42.171626+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:20:42.171626+0000","last_undegraded":"2024-09-21T01:20:42.171626+0000","last_fullsized":"2024-09-21T01:20:42.171626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169584+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:20:42.169584+0000","last_peered":"2024-09-21T01:20:42.169584+0000","last_clean":"2024-09-21T01:20:42.169584+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:20:42.169584+0000","last_undegraded":"2024-09-21T01:20:42.169584+0000","last_fullsized":"2024-09-21T01:20:42.169584+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73416,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":658,"num_read_kb":429,"num_write":250,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":67516,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":220248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":535509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1753088,"data_stored":1192890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037811,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1073152,"data_stored":535305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1056768,"data_stored":535549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32820,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712332,"statfs":{"total":95995035648,"available":95961427968,"internally_reserved":0,"allocated":1667072,"data_stored":1132571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":530809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790027,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1675264,"data_stored":1127762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1142784,"data_stored":602656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":530517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378715,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":531280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542240,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":598051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705766,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1073152,"data_stored":532265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:21:16.986 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T01:21:16.986 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T01:21:16.986 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T01:21:16.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T01:21:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:16 smithi089 ceph-mon[34486]: from='client.25171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:21:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:17.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:16 smithi098 ceph-mon[34372]: from='client.25171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:21:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:16 smithi027 ceph-mon[30375]: from='client.25171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:21:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2297126633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 msgr2=0x7f8b3810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2297126633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 0x7f8b3810bad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009900 tx=0x7f8b2002a410 comp rx=0 tx=0).stop 2024-09-21T01:21:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2297126633 shutdown_connections 2024-09-21T01:21:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2297126633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b3810c010 0x7f8b38112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2297126633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 0x7f8b3810bad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2297126633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b38106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2297126633 >> 172.21.15.27:0/2297126633 conn(0x7f8b38101810 msgr2=0x7f8b38103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2297126633 shutdown_connections 2024-09-21T01:21:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.145+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2297126633 wait complete. 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.146+0000 7f8b3dc00640 1 Processor -- start 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.146+0000 7f8b3dc00640 1 -- start start 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b3dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b3819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b3dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 0x7f8b3819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b3dc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b3810c010 0x7f8b381a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b3dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38115d20 con 0x7f8b3810b6c0 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b3dc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38115ba0 con 0x7f8b38105d80 2024-09-21T01:21:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b3dc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b38115ea0 con 0x7f8b3810c010 2024-09-21T01:21:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b36a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b3819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b2e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 0x7f8b3819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b36a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b3819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:58968/0 (socket says 172.21.15.27:58968) 2024-09-21T01:21:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b36a00640 1 -- 172.21.15.27:0/2350858134 learned_addr learned my addr 172.21.15.27:0/2350858134 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.147+0000 7f8b37400640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b3810c010 0x7f8b381a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b36a00640 1 -- 172.21.15.27:0/2350858134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b3810c010 msgr2=0x7f8b381a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b36a00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b3810c010 0x7f8b381a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b36a00640 1 -- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 msgr2=0x7f8b3819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b36a00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 0x7f8b3819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b36a00640 1 -- 172.21.15.27:0/2350858134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b381a2660 con 0x7f8b38105d80 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b36a00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b3819c6f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800e9f0 tx=0x7f8b2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800cd80 con 0x7f8b38105d80 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b381a2950 con 0x7f8b38105d80 2024-09-21T01:21:18.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b280108e0 con 0x7f8b38105d80 2024-09-21T01:21:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.148+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28018980 con 0x7f8b38105d80 2024-09-21T01:21:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.149+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381a90b0 con 0x7f8b38105d80 2024-09-21T01:21:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.149+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b38106190 con 0x7f8b38105d80 2024-09-21T01:21:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.152+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f8b28010430 con 0x7f8b38105d80 2024-09-21T01:21:18.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.154+0000 7f8b2fe00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8b1c0b65f0 0x7f8b1c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:18.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.154+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8b280e81d0 con 0x7f8b38105d80 2024-09-21T01:21:18.154 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.154+0000 7f8b2e000640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8b1c0b65f0 0x7f8b1c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:18.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.155+0000 7f8b2e000640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8b1c0b65f0 0x7f8b1c0b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b2002aa00 tx=0x7f8b20005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:18.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.155+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b280b27f0 con 0x7f8b38105d80 2024-09-21T01:21:18.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:17 smithi089 ceph-mon[34486]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:18.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.395+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8b3810de10 con 0x7f8b38105d80 2024-09-21T01:21:18.396 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:17 smithi027 ceph-mon[30375]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:18.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.395+0000 7f8b2fe00640 1 -- 172.21.15.27:0/2350858134 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8b280b7730 con 0x7f8b38105d80 2024-09-21T01:21:18.397 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:18.397 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T01:21:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:17 smithi098 ceph-mon[34372]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8b1c0b65f0 msgr2=0x7f8b1c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8b1c0b65f0 0x7f8b1c0b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8b2002aa00 tx=0x7f8b20005a20 comp rx=0 tx=0).stop 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 msgr2=0x7f8b3819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b3819c6f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800e9f0 tx=0x7f8b2800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 shutdown_connections 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8b1c0b65f0 0x7f8b1c0b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b3810c010 0x7f8b381a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b3810b6c0 0x7f8b3819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 --2- 172.21.15.27:0/2350858134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8b38105d80 0x7f8b3819c6f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.398+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 >> 172.21.15.27:0/2350858134 conn(0x7f8b38101810 msgr2=0x7f8b38103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.399+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 shutdown_connections 2024-09-21T01:21:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:18.399+0000 7f8b3dc00640 1 -- 172.21.15.27:0/2350858134 wait complete. 2024-09-21T01:21:19.066 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:18 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2350858134' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T01:21:19.103 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T01:21:19.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:21:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:18 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2350858134' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T01:21:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:18 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2350858134' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T01:21:19.490 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:20.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 -- 172.21.15.27:0/1293850969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120110440 msgr2=0x7ff120116c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:20.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 --2- 172.21.15.27:0/1293850969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120110440 0x7ff120116c70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff118009ca0 tx=0x7ff11802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 -- 172.21.15.27:0/1293850969 shutdown_connections 2024-09-21T01:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 --2- 172.21.15.27:0/1293850969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff120110440 0x7ff120116c70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 --2- 172.21.15.27:0/1293850969 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff12010faf0 0x7ff12010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 --2- 172.21.15.27:0/1293850969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff12010a1b0 0x7ff12010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 -- 172.21.15.27:0/1293850969 >> 172.21.15.27:0/1293850969 conn(0x7ff1200780e0 msgr2=0x7ff1201081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.162+0000 7ff126c00640 1 -- 172.21.15.27:0/1293850969 shutdown_connections 2024-09-21T01:21:20.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.163+0000 7ff126c00640 1 -- 172.21.15.27:0/1293850969 wait complete. 2024-09-21T01:21:20.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 Processor -- start 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 -- start start 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff12010a1b0 0x7ff1201a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 0x7ff1201a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff120110440 0x7ff1201a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff12011a110 con 0x7ff12010faf0 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff120119f90 con 0x7ff120110440 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff126c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff12011a290 con 0x7ff12010a1b0 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.164+0000 7ff11f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 0x7ff1201a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff11f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 0x7ff1201a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57586/0 (socket says 172.21.15.27:57586) 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff11f400640 1 -- 172.21.15.27:0/3849128882 learned_addr learned my addr 172.21.15.27:0/3849128882 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff120110440 0x7ff1201a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:20.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff11fe00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff12010a1b0 0x7ff1201a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 -- 172.21.15.27:0/3849128882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff12010a1b0 msgr2=0x7ff1201a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff12010a1b0 0x7ff1201a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 -- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 msgr2=0x7ff1201a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 0x7ff1201a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 -- 172.21.15.27:0/3849128882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1201a69f0 con 0x7ff120110440 2024-09-21T01:21:20.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff11f400640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 0x7ff1201a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.165+0000 7ff124c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff120110440 0x7ff1201a62c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff11802ade0 tx=0x7ff118031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.166+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118003b70 con 0x7ff120110440 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.166+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1201a6c80 con 0x7ff120110440 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.166+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff118031910 con 0x7ff120110440 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.166+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11802f950 con 0x7ff120110440 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.166+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201ad440 con 0x7ff120110440 2024-09-21T01:21:20.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.167+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12010a5c0 con 0x7ff120110440 2024-09-21T01:21:20.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.169+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ff118004b40 con 0x7ff120110440 2024-09-21T01:21:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.171+0000 7ff11cc00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1040b65f0 0x7ff1040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:20.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.171+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff118108bd0 con 0x7ff120110440 2024-09-21T01:21:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.171+0000 7ff11fe00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1040b65f0 0x7ff1040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:20.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.172+0000 7ff11fe00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1040b65f0 0x7ff1040b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff110009bd0 tx=0x7ff110005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:20.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.173+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1180cee10 con 0x7ff120110440 2024-09-21T01:21:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:19 smithi089 ceph-mon[34486]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:19 smithi098 ceph-mon[34372]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:20.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.412+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1201a1bb0 con 0x7ff120110440 2024-09-21T01:21:20.413 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:19 smithi027 ceph-mon[30375]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:20.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.413+0000 7ff11cc00640 1 -- 172.21.15.27:0/3849128882 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7ff1201a1bb0 con 0x7ff120110440 2024-09-21T01:21:20.415 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:20.415 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T01:20:08:397428+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:20:07.570104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24718,24730,24734,24752,24776]},"id":1}]} 2024-09-21T01:21:20.415 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.415+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1040b65f0 msgr2=0x7ff1040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.415+0000 7ff126c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1040b65f0 0x7ff1040b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff110009bd0 tx=0x7ff110005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.415+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff120110440 msgr2=0x7ff1201a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.415+0000 7ff126c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff120110440 0x7ff1201a62c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff11802ade0 tx=0x7ff118031660 comp rx=0 tx=0).stop 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 shutdown_connections 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1040b65f0 0x7ff1040b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff120110440 0x7ff1201a62c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff12010faf0 0x7ff1201a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 --2- 172.21.15.27:0/3849128882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff12010a1b0 0x7ff1201a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:20.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 >> 172.21.15.27:0/3849128882 conn(0x7ff1200780e0 msgr2=0x7ff120078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 shutdown_connections 2024-09-21T01:21:20.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:20.416+0000 7ff126c00640 1 -- 172.21.15.27:0/3849128882 wait complete. 2024-09-21T01:21:21.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:20 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3849128882' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:21.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:21:21.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:20 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3849128882' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:20 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3849128882' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:21.621 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:22.123 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:21 smithi027 ceph-mon[30375]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:21 smithi089 ceph-mon[34486]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 -- 172.21.15.27:0/2772510255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 msgr2=0x7efcbc0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 --2- 172.21.15.27:0/2772510255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 0x7efcbc0757b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009860 tx=0x7efcb002a340 comp rx=0 tx=0).stop 2024-09-21T01:21:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 -- 172.21.15.27:0/2772510255 shutdown_connections 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 --2- 172.21.15.27:0/2772510255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcbc114440 0x7efcbc116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 --2- 172.21.15.27:0/2772510255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 --2- 172.21.15.27:0/2772510255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 0x7efcbc0757b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 -- 172.21.15.27:0/2772510255 >> 172.21.15.27:0/2772510255 conn(0x7efcbc0ff5e0 msgr2=0x7efcbc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 -- 172.21.15.27:0/2772510255 shutdown_connections 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.306+0000 7efcc3600640 1 -- 172.21.15.27:0/2772510255 wait complete. 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.307+0000 7efcc3600640 1 Processor -- start 2024-09-21T01:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 -- start start 2024-09-21T01:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 0x7efcbc1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcbc114440 0x7efcbc1a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc11a070 con 0x7efcbc075dd0 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc119ef0 con 0x7efcbc077380 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.308+0000 7efcc3600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc11a1f0 con 0x7efcbc114440 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc1a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57616/0 (socket says 172.21.15.27:57616) 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 -- 172.21.15.27:0/2162726682 learned_addr learned my addr 172.21.15.27:0/2162726682 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcbbe00640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 0x7efcbc1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc1600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcbc114440 0x7efcbc1a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 -- 172.21.15.27:0/2162726682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcbc114440 msgr2=0x7efcbc1a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcbc114440 0x7efcbc1a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 -- 172.21.15.27:0/2162726682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 msgr2=0x7efcbc1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 0x7efcbc1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.309+0000 7efcc0c00640 1 -- 172.21.15.27:0/2162726682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc1a69f0 con 0x7efcbc075dd0 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.310+0000 7efcc0c00640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc1a0a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efcb0035040 tx=0x7efcb0032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.310+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0037040 con 0x7efcbc075dd0 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.310+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcbc1a6c80 con 0x7efcbc075dd0 2024-09-21T01:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.310+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb0041070 con 0x7efcbc075dd0 2024-09-21T01:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.310+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00035f0 con 0x7efcbc075dd0 2024-09-21T01:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.310+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcbc1a7100 con 0x7efcbc075dd0 2024-09-21T01:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.312+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcbc0761e0 con 0x7efcbc075dd0 2024-09-21T01:21:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.314+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7efcb0004050 con 0x7efcbc075dd0 2024-09-21T01:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.316+0000 7efcb9600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efca40b65f0 0x7efca40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.316+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efcb01073a0 con 0x7efcbc075dd0 2024-09-21T01:21:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.316+0000 7efcbbe00640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efca40b65f0 0x7efca40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.317+0000 7efcbbe00640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efca40b65f0 0x7efca40b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efcbc116590 tx=0x7efca8009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.321+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efcb00cd640 con 0x7efcbc075dd0 2024-09-21T01:21:22.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:21 smithi098 ceph-mon[34372]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:22.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.563+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcbc02e640 con 0x7efcbc075dd0 2024-09-21T01:21:22.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.564+0000 7efcb9600640 1 -- 172.21.15.27:0/2162726682 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7efcb00d1a20 con 0x7efcbc075dd0 2024-09-21T01:21:22.566 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:22.566 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T01:20:08:397428+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:20:07.570104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24718,24730,24734,24752,24776]},"id":1}]} 2024-09-21T01:21:22.566 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:21:22.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efca40b65f0 msgr2=0x7efca40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efca40b65f0 0x7efca40b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efcbc116590 tx=0x7efca8009260 comp rx=0 tx=0).stop 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 msgr2=0x7efcbc1a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc1a0a80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efcb0035040 tx=0x7efcb0032d50 comp rx=0 tx=0).stop 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 shutdown_connections 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efca40b65f0 0x7efca40b8b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcbc114440 0x7efcbc1a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efcbc077380 0x7efcbc1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 --2- 172.21.15.27:0/2162726682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcbc075dd0 0x7efcbc1a0a80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 >> 172.21.15.27:0/2162726682 conn(0x7efcbc0ff5e0 msgr2=0x7efcbc110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 shutdown_connections 2024-09-21T01:21:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:22.567+0000 7efcc3600640 1 -- 172.21.15.27:0/2162726682 wait complete. 2024-09-21T01:21:23.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:22 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2162726682' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:23.379 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:22 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2162726682' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:23.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:22 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2162726682' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:23.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:21:23.805 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:23 smithi089 ceph-mon[34486]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:24.283 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:23 smithi027 ceph-mon[30375]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:24.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:23 smithi098 ceph-mon[34372]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:24.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 -- 172.21.15.27:0/2463104971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834076080 msgr2=0x7fc83410fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 --2- 172.21.15.27:0/2463104971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834076080 0x7fc83410fa50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc820009860 tx=0x7fc82002a340 comp rx=0 tx=0).stop 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 -- 172.21.15.27:0/2463104971 shutdown_connections 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 --2- 172.21.15.27:0/2463104971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83410ff90 0x7fc8341123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 --2- 172.21.15.27:0/2463104971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834076080 0x7fc83410fa50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 --2- 172.21.15.27:0/2463104971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc834075730 0x7fc834075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 -- 172.21.15.27:0/2463104971 >> 172.21.15.27:0/2463104971 conn(0x7fc8340ff400 msgr2=0x7fc834101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 -- 172.21.15.27:0/2463104971 shutdown_connections 2024-09-21T01:21:24.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.467+0000 7fc838c00640 1 -- 172.21.15.27:0/2463104971 wait complete. 2024-09-21T01:21:24.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.468+0000 7fc838c00640 1 Processor -- start 2024-09-21T01:21:24.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.468+0000 7fc838c00640 1 -- start start 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.468+0000 7fc838c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 0x7fc83419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.468+0000 7fc838c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc834076080 0x7fc83419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.468+0000 7fc838c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83410ff90 0x7fc8341a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc838c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc834115ad0 con 0x7fc834076080 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc838c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc834115950 con 0x7fc83410ff90 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc838c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc834115c50 con 0x7fc834075730 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 0x7fc83419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc832a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc834076080 0x7fc83419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 0x7fc83419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:54288/0 (socket says 172.21.15.27:54288) 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 -- 172.21.15.27:0/4003645296 learned_addr learned my addr 172.21.15.27:0/4003645296 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:24.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833e00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83410ff90 0x7fc8341a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:24.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 -- 172.21.15.27:0/4003645296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83410ff90 msgr2=0x7fc8341a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:24.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83410ff90 0x7fc8341a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 -- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc834076080 msgr2=0x7fc83419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:24.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc834076080 0x7fc83419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.469+0000 7fc833400640 1 -- 172.21.15.27:0/4003645296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8341a2430 con 0x7fc834075730 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.470+0000 7fc833400640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 0x7fc83419c4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc82400e9c0 tx=0x7fc82400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.470+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400ccc0 con 0x7fc834075730 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.470+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8240054d0 con 0x7fc834075730 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.470+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8341a2720 con 0x7fc834075730 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.470+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824010430 con 0x7fc834075730 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.470+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8341a8e80 con 0x7fc834075730 2024-09-21T01:21:24.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.471+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc834075b40 con 0x7fc834075730 2024-09-21T01:21:24.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.474+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc824004f80 con 0x7fc834075730 2024-09-21T01:21:24.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.476+0000 7fc82be00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc8180b65f0 0x7fc8180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:24.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.476+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fc8240e80c0 con 0x7fc834075730 2024-09-21T01:21:24.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.476+0000 7fc832a00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc8180b65f0 0x7fc8180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:24.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.477+0000 7fc832a00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc8180b65f0 0x7fc8180b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc820005940 tx=0x7fc8200058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:24.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.477+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8240ae300 con 0x7fc834075730 2024-09-21T01:21:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.677+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc83419d5f0 con 0x7fc834075730 2024-09-21T01:21:24.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.679+0000 7fc82be00640 1 -- 172.21.15.27:0/4003645296 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25150 (secure 0 0 0) 0x7fc83419d5f0 con 0x7fc834075730 2024-09-21T01:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:24.681 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:20:42.161651+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc8180b65f0 msgr2=0x7fc8180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc8180b65f0 0x7fc8180b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc820005940 tx=0x7fc8200058d0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 msgr2=0x7fc83419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 0x7fc83419c4c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc82400e9c0 tx=0x7fc82400ee80 comp rx=0 tx=0).stop 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 shutdown_connections 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc8180b65f0 0x7fc8180b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc83410ff90 0x7fc8341a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc834076080 0x7fc83419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 --2- 172.21.15.27:0/4003645296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc834075730 0x7fc83419c4c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 >> 172.21.15.27:0/4003645296 conn(0x7fc8340ff400 msgr2=0x7fc834109b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 shutdown_connections 2024-09-21T01:21:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:24.682+0000 7fc838c00640 1 -- 172.21.15.27:0/4003645296 wait complete. 2024-09-21T01:21:25.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:25 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4003645296' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:21:25.405 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:25 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4003645296' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:21:25.444 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:21:25.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:25 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4003645296' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:21:25.831 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:26.325 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:26 smithi027 ceph-mon[30375]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:26 smithi098 ceph-mon[34372]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:26.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:26 smithi089 ceph-mon[34486]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:26.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.507+0000 7f4eee000640 1 -- 172.21.15.27:0/319090877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee8075750 msgr2=0x7f4ee8075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:26.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.507+0000 7f4eee000640 1 --2- 172.21.15.27:0/319090877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee8075750 0x7f4ee8075b60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009900 tx=0x7f4edc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 -- 172.21.15.27:0/319090877 shutdown_connections 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 --2- 172.21.15.27:0/319090877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee810ff70 0x7f4ee8112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 --2- 172.21.15.27:0/319090877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80760a0 0x7f4ee810fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 --2- 172.21.15.27:0/319090877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee8075750 0x7f4ee8075b60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 -- 172.21.15.27:0/319090877 >> 172.21.15.27:0/319090877 conn(0x7f4ee80ff400 msgr2=0x7f4ee8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 -- 172.21.15.27:0/319090877 shutdown_connections 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.508+0000 7f4eee000640 1 -- 172.21.15.27:0/319090877 wait complete. 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 Processor -- start 2024-09-21T01:21:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 -- start start 2024-09-21T01:21:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 0x7f4ee819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee80760a0 0x7f4ee819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee810ff70 0x7f4ee81a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8115b30 con 0x7f4ee8075750 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81159b0 con 0x7f4ee810ff70 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.509+0000 7f4eee000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8115cb0 con 0x7f4ee80760a0 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 0x7f4ee819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 0x7f4ee819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47248/0 (socket says 172.21.15.27:47248) 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 -- 172.21.15.27:0/2636704374 learned_addr learned my addr 172.21.15.27:0/2636704374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eed600640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee810ff70 0x7f4ee81a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4ee7e00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee80760a0 0x7f4ee819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 -- 172.21.15.27:0/2636704374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee80760a0 msgr2=0x7f4ee819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee80760a0 0x7f4ee819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 -- 172.21.15.27:0/2636704374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee810ff70 msgr2=0x7f4ee81a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee810ff70 0x7f4ee81a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4eecc00640 1 -- 172.21.15.27:0/2636704374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee81a2430 con 0x7f4ee8075750 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.510+0000 7f4ee7e00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee80760a0 0x7f4ee819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.511+0000 7f4eecc00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 0x7f4ee819c4c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4edc005ab0 tx=0x7f4edc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.511+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc03f070 con 0x7f4ee8075750 2024-09-21T01:21:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.511+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4edc031b60 con 0x7f4ee8075750 2024-09-21T01:21:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.511+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81a26c0 con 0x7f4ee8075750 2024-09-21T01:21:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.511+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc035030 con 0x7f4ee8075750 2024-09-21T01:21:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.511+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a8e80 con 0x7f4ee8075750 2024-09-21T01:21:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.512+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee8076c60 con 0x7f4ee8075750 2024-09-21T01:21:26.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.514+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f4edc042070 con 0x7f4ee8075750 2024-09-21T01:21:26.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.516+0000 7f4ee5600640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4ed00b65f0 0x7f4ed00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:26.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.516+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4edc1080b0 con 0x7f4ee8075750 2024-09-21T01:21:26.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.516+0000 7f4ee7e00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4ed00b65f0 0x7f4ed00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:26.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.517+0000 7f4ee7e00640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4ed00b65f0 0x7f4ed00b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8112300 tx=0x7f4ed4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:26.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.521+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4edc0ce2f0 con 0x7f4ee8075750 2024-09-21T01:21:26.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.759+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ee81a9050 con 0x7f4ee8075750 2024-09-21T01:21:26.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.760+0000 7f4ee5600640 1 -- 172.21.15.27:0/2636704374 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f4edc0d26d0 con 0x7f4ee8075750 2024-09-21T01:21:26.762 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:26.762 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T01:20:08:397428+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:20:07.570104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24718,24730,24734,24752,24776]},"id":1}]} 2024-09-21T01:21:26.762 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:21:26.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4ed00b65f0 msgr2=0x7f4ed00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4ed00b65f0 0x7f4ed00b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8112300 tx=0x7f4ed4005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 msgr2=0x7f4ee819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 0x7f4ee819c4c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4edc005ab0 tx=0x7f4edc003c10 comp rx=0 tx=0).stop 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 shutdown_connections 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4ed00b65f0 0x7f4ed00b8b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4ee810ff70 0x7f4ee81a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ee80760a0 0x7f4ee819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 --2- 172.21.15.27:0/2636704374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee8075750 0x7f4ee819c4c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.762+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 >> 172.21.15.27:0/2636704374 conn(0x7f4ee80ff400 msgr2=0x7f4ee8109b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.763+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 shutdown_connections 2024-09-21T01:21:26.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:26.763+0000 7f4eee000640 1 -- 172.21.15.27:0/2636704374 wait complete. 2024-09-21T01:21:27.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:27 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2636704374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:27.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:27 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2636704374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:27.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:27 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2636704374' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:27.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T01:19:07.935504+0000', 'modified': '2024-09-21T01:20:07.570104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24776, 'mds_1': 24752, 'mds_2': 24734, 'mds_3': 24730, 'mds_4': 24718}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6833/39744273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6832', 'nonce': 39744273}, {'type': 'v1', 'addr': '172.21.15.89:6833', 'nonce': 39744273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24686': {'gid': 24686, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6835/2864798916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 2864798916}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2864798916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6837/392681445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6836', 'nonce': 392681445}, {'type': 'v1', 'addr': '172.21.15.27:6837', 'nonce': 392681445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24706': {'gid': 24706, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1764678372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1764678372}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1764678372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6835/3850699909', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 3850699909}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 3850699909}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24718': {'gid': 24718, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.98:6837/1081631404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1081631404}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1081631404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24730': {'gid': 24730, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.98:6839/3799755243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3799755243}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3799755243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24734': {'gid': 24734, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.89:6837/3556269004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3556269004}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3556269004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24752': {'gid': 24752, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.27:6839/3115497172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3115497172}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3115497172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24776': {'gid': 24776, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4156411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4156411896}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4156411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24776, 'qdb_cluster': [24718, 24730, 24734, 24752, 24776]} 2024-09-21T01:21:27.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T01:21:27.564 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T01:21:27.949 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:28.279 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:28 smithi027 ceph-mon[30375]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:28 smithi098 ceph-mon[34372]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:28 smithi089 ceph-mon[34486]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:28.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 -- 172.21.15.27:0/1671225569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 msgr2=0x7fc20c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 --2- 172.21.15.27:0/1671225569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c1098b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009900 tx=0x7fc1f402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 -- 172.21.15.27:0/1671225569 shutdown_connections 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 --2- 172.21.15.27:0/1671225569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc20c109df0 0x7fc20c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 --2- 172.21.15.27:0/1671225569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c1098b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 --2- 172.21.15.27:0/1671225569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c103b60 0x7fc20c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 -- 172.21.15.27:0/1671225569 >> 172.21.15.27:0/1671225569 conn(0x7fc20c07b850 msgr2=0x7fc20c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 -- 172.21.15.27:0/1671225569 shutdown_connections 2024-09-21T01:21:28.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.651+0000 7fc211400640 1 -- 172.21.15.27:0/1671225569 wait complete. 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.652+0000 7fc211400640 1 Processor -- start 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 -- start start 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c103b60 0x7fc20c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc20c109df0 0x7fc20c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c113b00 con 0x7fc20c109df0 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c113980 con 0x7fc20c1094a0 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc211400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c113c80 con 0x7fc20c103b60 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc203e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc20b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc20c109df0 0x7fc20c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc203e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57460/0 (socket says 172.21.15.27:57460) 2024-09-21T01:21:28.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.653+0000 7fc203e00640 1 -- 172.21.15.27:0/2032267336 learned_addr learned my addr 172.21.15.27:0/2032267336 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc20a800640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c103b60 0x7fc20c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc203e00640 1 -- 172.21.15.27:0/2032267336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c103b60 msgr2=0x7fc20c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc203e00640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c103b60 0x7fc20c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc203e00640 1 -- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc20c109df0 msgr2=0x7fc20c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc203e00640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc20c109df0 0x7fc20c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc203e00640 1 -- 172.21.15.27:0/2032267336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c1a8770 con 0x7fc20c1094a0 2024-09-21T01:21:28.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.654+0000 7fc203e00640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c075c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005ab0 tx=0x7fc1f4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.655+0000 7fc203400640 1 -- 172.21.15.27:0/2032267336 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f403f070 con 0x7fc20c1094a0 2024-09-21T01:21:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.655+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c1a8a00 con 0x7fc20c1094a0 2024-09-21T01:21:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.655+0000 7fc203400640 1 -- 172.21.15.27:0/2032267336 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4031b50 con 0x7fc20c1094a0 2024-09-21T01:21:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.655+0000 7fc203400640 1 -- 172.21.15.27:0/2032267336 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4035030 con 0x7fc20c1094a0 2024-09-21T01:21:28.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.655+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c1a8f30 con 0x7fc20c1094a0 2024-09-21T01:21:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 Processor -- start 2024-09-21T01:21:28.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 -- start start 2024-09-21T01:21:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 0x5652d5aec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5aec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5725280 0x5652d5aecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5652d5a63d50 con 0x5652d5745f60 2024-09-21T01:21:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5652d5a53180 con 0x5652d571d0a0 2024-09-21T01:21:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.658+0000 7fc217fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5652d5a56f90 con 0x5652d5725280 2024-09-21T01:21:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 0x5652d5aec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 0x5652d5aec490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47288/0 (socket says 172.21.15.27:47288) 2024-09-21T01:21:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 -- 172.21.15.27:0/2493691111 learned_addr learned my addr 172.21.15.27:0/2493691111 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20a800640 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5aec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 -- 172.21.15.27:0/2493691111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5725280 msgr2=0x5652d5aecf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5725280 0x5652d5aecf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 -- 172.21.15.27:0/2493691111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 msgr2=0x5652d5aec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5aec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.659+0000 7fc20b200640 1 -- 172.21.15.27:0/2493691111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d5a1da70 con 0x5652d5745f60 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.660+0000 7fc203400640 1 -- 172.21.15.27:0/2032267336 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fc1f4042070 con 0x7fc20c1094a0 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.660+0000 7fc20b200640 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 0x5652d5aec490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc204009f50 tx=0x7fc2040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ca00eb7edd8319 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.660+0000 7fc1fb400640 1 -- 172.21.15.27:0/2493691111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204004b20 con 0x5652d5745f60 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.660+0000 7fc1fb400640 1 -- 172.21.15.27:0/2493691111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc204003e20 con 0x5652d5745f60 2024-09-21T01:21:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.660+0000 7fc1fb400640 1 -- 172.21.15.27:0/2493691111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040047e0 con 0x5652d5745f60 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 -- 172.21.15.27:0/2493691111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 msgr2=0x5652d5aec490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 0x5652d5aec490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc204009f50 tx=0x7fc2040118d0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 -- 172.21.15.27:0/2493691111 shutdown_connections 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5725280 0x5652d5aecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5aec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2493691111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5745f60 0x5652d5aec490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 -- 172.21.15.27:0/2493691111 >> 172.21.15.27:0/2493691111 conn(0x5652d573a350 msgr2=0x5652d5aea380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 -- 172.21.15.27:0/2493691111 shutdown_connections 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.661+0000 7fc217fd2740 1 -- 172.21.15.27:0/2493691111 wait complete. 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.662+0000 7fc203400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc1f00b6500 0x7fc1f00b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.662+0000 7fc217fd2740 1 -- start start 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.662+0000 7fc203400640 1 -- 172.21.15.27:0/2032267336 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fc1f4107f90 con 0x7fc20c1094a0 2024-09-21T01:21:28.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.662+0000 7fc20b200640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc1f00b6500 0x7fc1f00b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.662+0000 7fc217fd2740 1 Processor -- start 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 0x5652d5af80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d5725280 0x5652d5af85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc20b200640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc1f00b6500 0x7fc1f00b8a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc204009250 tx=0x7fc204000f40 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b03f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5652d58b45f0 con 0x5652d571d0a0 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5652d59ee460 con 0x5652d5725280 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5652d5afac70 con 0x5652d5745f60 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc217fd2740 10 client.0 _ll_register_callbacks cb 0x5652d59fe210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc20a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 0x5652d5af80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc20a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 0x5652d5af80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47294/0 (socket says 172.21.15.27:47294) 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc20a800640 1 -- 172.21.15.27:0/2871621137 learned_addr learned my addr 172.21.15.27:0/2871621137 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc203e00640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d5725280 0x5652d5af85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.663+0000 7fc20b200640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b03f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc20a800640 1 -- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 msgr2=0x5652d5b03f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc20a800640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b03f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc20a800640 1 -- 172.21.15.27:0/2871621137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d5725280 msgr2=0x5652d5af85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc20a800640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d5725280 0x5652d5af85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc20a800640 1 -- 172.21.15.27:0/2871621137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d57b1540 con 0x5652d571d0a0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc20a800640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 0x5652d5af80b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc012710 tx=0x7fc1fc012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d86be87bc8874f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc1efe00640 1 -- 172.21.15.27:0/2871621137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004030 con 0x5652d571d0a0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc1efe00640 1 -- 172.21.15.27:0/2871621137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fc1fc0041d0 con 0x5652d571d0a0 2024-09-21T01:21:28.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.664+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652d59f0a70 con 0x5652d571d0a0 2024-09-21T01:21:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc1efe00640 1 -- 172.21.15.27:0/2871621137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1fc025050 con 0x5652d571d0a0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 10 client.15180 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652d5a0b6f0 con 0x5652d571d0a0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc1efe00640 1 -- 172.21.15.27:0/2871621137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5652d59f0a70 con 0x5652d571d0a0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc1efe00640 1 -- 172.21.15.27:0/2871621137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc1fc0039c0 con 0x5652d571d0a0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 10 client.15180 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 10 client.15180 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 20 client.15180 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d57048e0 0x5652d5b0a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 4 client.15180 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.665+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 --> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652d5a65ca0 con 0x5652d57048e0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.666+0000 7fc203e00640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d57048e0 0x5652d5b0a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.666+0000 7fc203e00640 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d57048e0 0x5652d5b0a560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6df03aa8b576e738 server_cookie=8520003adb605481 in_seq=0 out_seq=0 2024-09-21T01:21:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.666+0000 7fc1efe00640 10 client.15180 ms_handle_connect on v2:172.21.15.98:6836/1081631404 2024-09-21T01:21:28.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.667+0000 7fc1efe00640 1 -- 172.21.15.27:0/2871621137 <== mds.4 v2:172.21.15.98:6836/1081631404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5652d5a65ca0 con 0x5652d57048e0 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.667+0000 7fc1efe00640 10 client.15180 handle_command_reply: tid=0 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.667+0000 7fc217fd2740 1 client.15180 shutdown 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 msgr2=0x5652d5af80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 0x5652d5af80b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc012710 tx=0x7fc1fc012bd0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 shutdown_connections 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d57048e0 0x5652d5b0a560 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b03f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc1efe00640 0 client.15180 ms_handle_reset on v2:172.21.15.98:6836/1081631404 2024-09-21T01:21:28.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d5725280 0x5652d5af85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 --2- 172.21.15.27:0/2871621137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d571d0a0 0x5652d5af80b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 >> 172.21.15.27:0/2871621137 conn(0x5652d573a350 msgr2=0x5652d5aef670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 shutdown_connections 2024-09-21T01:21:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 1 -- 172.21.15.27:0/2871621137 wait complete. 2024-09-21T01:21:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.668+0000 7fc217fd2740 20 client.15180 trim_cache size 0 max 16384 2024-09-21T01:21:28.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.687+0000 7fc217fd2740 1 Processor -- start 2024-09-21T01:21:28.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.687+0000 7fc217fd2740 1 -- start start 2024-09-21T01:21:28.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5ae5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc217fd2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5652d5b088a0 con 0x5652d5725280 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc217fd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5652d5b08f70 con 0x5652d571d0a0 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc217fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5652d5b09640 con 0x5652d5745f60 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b03ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47296/0 (socket says 172.21.15.27:47296) 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 -- 172.21.15.27:0/1278055557 learned_addr learned my addr 172.21.15.27:0/1278055557 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc20b200640 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5ae5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc20a800640 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 -- 172.21.15.27:0/1278055557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 msgr2=0x5652d5ae5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5ae5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 -- 172.21.15.27:0/1278055557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 msgr2=0x5652d5b0a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.688+0000 7fc203e00640 1 -- 172.21.15.27:0/1278055557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d5aed2a0 con 0x5652d5725280 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.689+0000 7fc203e00640 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b03ed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009320 tx=0x7fc1f403b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7a877504bd5ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.689+0000 7fc1faa00640 1 -- 172.21.15.27:0/1278055557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4049040 con 0x5652d5725280 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.689+0000 7fc1faa00640 1 -- 172.21.15.27:0/1278055557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40cd690 con 0x5652d5725280 2024-09-21T01:21:28.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.689+0000 7fc1faa00640 1 -- 172.21.15.27:0/1278055557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f41056a0 con 0x5652d5725280 2024-09-21T01:21:28.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 -- 172.21.15.27:0/1278055557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 msgr2=0x5652d5b03ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b03ed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009320 tx=0x7fc1f403b6b0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 -- 172.21.15.27:0/1278055557 shutdown_connections 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5ae5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b03ed0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 --2- 172.21.15.27:0/1278055557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 -- 172.21.15.27:0/1278055557 >> 172.21.15.27:0/1278055557 conn(0x5652d573a350 msgr2=0x5652d5af3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 -- 172.21.15.27:0/1278055557 shutdown_connections 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.690+0000 7fc217fd2740 1 -- 172.21.15.27:0/1278055557 wait complete. 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 -- start start 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 Processor -- start 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b0bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5652d5b0a3e0 con 0x5652d5725280 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5652d5b09d10 con 0x5652d571d0a0 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5652d5b0aab0 con 0x5652d5745f60 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.691+0000 7fc217fd2740 10 client.0 _ll_register_callbacks cb 0x7fc1fc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b0bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b0bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47310/0 (socket says 172.21.15.27:47310) 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 -- 172.21.15.27:0/3224922500 learned_addr learned my addr 172.21.15.27:0/3224922500 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc20a800640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc20b200640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 -- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 msgr2=0x5652d5b0c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b0c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 -- 172.21.15.27:0/3224922500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 msgr2=0x5652d5b0b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 -- 172.21.15.27:0/3224922500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d5b12800 con 0x5652d5725280 2024-09-21T01:21:28.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc20a800640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc203e00640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b0bbb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4031440 tx=0x7fc1f41053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9066035671349a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.692+0000 7fc1efe00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc1efe00640 1 -- 172.21.15.27:0/3224922500 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4049070 con 0x5652d5725280 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc1efe00640 1 -- 172.21.15.27:0/3224922500 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fc1f40f1070 con 0x5652d5725280 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f40cd690 con 0x5652d5725280 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc1efe00640 1 -- 172.21.15.27:0/3224922500 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1f40d25b0 con 0x5652d5725280 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc217fd2740 10 client.15186 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652d5b75d20 con 0x5652d5725280 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc1efe00640 1 -- 172.21.15.27:0/3224922500 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40cd690 con 0x5652d5725280 2024-09-21T01:21:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc1efe00640 1 -- 172.21.15.27:0/3224922500 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc1f403bc40 con 0x5652d5725280 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.693+0000 7fc217fd2740 10 client.15186 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc217fd2740 10 client.15186 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc217fd2740 20 client.15186 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc217fd2740 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d56d9c90 0x5652d5b79330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc217fd2740 4 client.15186 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 --> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] -- command(tid 0: {"prefix": "status"}) -- 0x5652d5a61750 con 0x5652d56d9c90 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc20a800640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d56d9c90 0x5652d5b79330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc20a800640 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d56d9c90 0x5652d5b79330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d008030a3bf8729 server_cookie=53726b83ac1a030 in_seq=0 out_seq=0 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.694+0000 7fc1efe00640 10 client.15186 ms_handle_connect on v2:172.21.15.98:6836/1081631404 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc1efe00640 1 -- 172.21.15.27:0/3224922500 <== mds.4 v2:172.21.15.98:6836/1081631404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5652d5a61750 con 0x5652d56d9c90 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc1efe00640 10 client.15186 handle_command_reply: tid=0 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 client.15186 shutdown 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 msgr2=0x5652d5b0bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b0bbb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4031440 tx=0x7fc1f41053c0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 shutdown_connections 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5652d56d9c90 0x5652d5b79330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5652d5745f60 0x5652d5b0c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5652d5725280 0x5652d5b0bbb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 --2- 172.21.15.27:0/3224922500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5652d571d0a0 0x5652d5b0b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 >> 172.21.15.27:0/3224922500 conn(0x5652d573a350 msgr2=0x5652d5aed4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc1efe00640 0 client.15186 ms_handle_reset on v2:172.21.15.98:6836/1081631404 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 shutdown_connections 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 1 -- 172.21.15.27:0/3224922500 wait complete. 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.695+0000 7fc217fd2740 20 client.15186 trim_cache size 0 max 16384 2024-09-21T01:21:28.699 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24718, 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 84.320591045, 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 170.639701457 2024-09-21T01:21:28.700 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:21:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc1f00b6500 msgr2=0x7fc1f00b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc1f00b6500 0x7fc1f00b8a10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc204009250 tx=0x7fc204000f40 comp rx=0 tx=0).stop 2024-09-21T01:21:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 msgr2=0x7fc20c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c075c50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005ab0 tx=0x7fc1f4003b90 comp rx=0 tx=0).stop 2024-09-21T01:21:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 shutdown_connections 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc1f00b6500 0x7fc1f00b8a10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc20c109df0 0x7fc20c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc20c1094a0 0x7fc20c075c50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 --2- 172.21.15.27:0/2032267336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc20c103b60 0x7fc20c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 >> 172.21.15.27:0/2032267336 conn(0x7fc20c07b850 msgr2=0x7fc20c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 shutdown_connections 2024-09-21T01:21:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:28.696+0000 7fc211400640 1 -- 172.21.15.27:0/2032267336 wait complete. 2024-09-21T01:21:29.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T01:21:29.908 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:30.353 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:30 smithi027 ceph-mon[30375]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:30 smithi098 ceph-mon[34372]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:30.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:30 smithi089 ceph-mon[34486]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:30.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 -- 172.21.15.27:0/263657454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449810b6c0 msgr2=0x7f449810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 --2- 172.21.15.27:0/263657454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449810b6c0 0x7f449810bad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4484009900 tx=0x7f448402a410 comp rx=0 tx=0).stop 2024-09-21T01:21:30.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 -- 172.21.15.27:0/263657454 shutdown_connections 2024-09-21T01:21:30.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 --2- 172.21.15.27:0/263657454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 0x7f4498112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 --2- 172.21.15.27:0/263657454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f449810b6c0 0x7f449810bad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 --2- 172.21.15.27:0/263657454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4498105d80 0x7f4498106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 -- 172.21.15.27:0/263657454 >> 172.21.15.27:0/263657454 conn(0x7f44981017d0 msgr2=0x7f4498103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:30.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 -- 172.21.15.27:0/263657454 shutdown_connections 2024-09-21T01:21:30.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.537+0000 7f449ec00640 1 -- 172.21.15.27:0/263657454 wait complete. 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.538+0000 7f449ec00640 1 Processor -- start 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.538+0000 7f449ec00640 1 -- start start 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449ec00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105d80 0x7f449819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449ec00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810b6c0 0x7f449819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449ec00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 0x7f44981a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449ec00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4498115d20 con 0x7f4498105d80 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449ec00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4498115ba0 con 0x7f449810c010 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449ec00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4498115ea0 con 0x7f449810b6c0 2024-09-21T01:21:30.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f4497e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105d80 0x7f449819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f449cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 0x7f44981a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f4497400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810b6c0 0x7f449819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f4497e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105d80 0x7f449819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47320/0 (socket says 172.21.15.27:47320) 2024-09-21T01:21:30.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.539+0000 7f4497e00640 1 -- 172.21.15.27:0/2077865467 learned_addr learned my addr 172.21.15.27:0/2077865467 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449cc00640 1 -- 172.21.15.27:0/2077865467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810b6c0 msgr2=0x7f449819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449cc00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810b6c0 0x7f449819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449cc00640 1 -- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105d80 msgr2=0x7f449819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449cc00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105d80 0x7f449819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449cc00640 1 -- 172.21.15.27:0/2077865467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981a2660 con 0x7f449810c010 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449cc00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 0x7f44981a1f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f449000ec50 tx=0x7f449000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f4494c00640 1 -- 172.21.15.27:0/2077865467 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490003b70 con 0x7f449810c010 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f4494c00640 1 -- 172.21.15.27:0/2077865467 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4490003d10 con 0x7f449810c010 2024-09-21T01:21:30.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f4494c00640 1 -- 172.21.15.27:0/2077865467 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490010830 con 0x7f449810c010 2024-09-21T01:21:30.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.540+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981a2950 con 0x7f449810c010 2024-09-21T01:21:30.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.541+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981a90b0 con 0x7f449810c010 2024-09-21T01:21:30.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.544+0000 7f44a5879740 1 Processor -- start 2024-09-21T01:21:30.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f4494c00640 1 -- 172.21.15.27:0/2077865467 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f4490003ec0 con 0x7f449810c010 2024-09-21T01:21:30.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 -- start start 2024-09-21T01:21:30.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7e5280 0x56299abacf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56299ab23d50 con 0x56299a7dd0a0 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56299ab13180 con 0x56299a805f60 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f44a5879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56299ab16f90 con 0x56299a7e5280 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.546+0000 7f4497e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47332/0 (socket says 172.21.15.27:47332) 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 -- 172.21.15.27:0/2262754456 learned_addr learned my addr 172.21.15.27:0/2262754456 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497400640 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497400640 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7e5280 0x56299abacf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 -- 172.21.15.27:0/2262754456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7e5280 msgr2=0x56299abacf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7e5280 0x56299abacf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 -- 172.21.15.27:0/2262754456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 msgr2=0x56299abac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 -- 172.21.15.27:0/2262754456 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299aadda70 con 0x56299a7dd0a0 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497400640 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7e5280 0x56299abacf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4494c00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f447c0b65f0 0x7f447c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.547+0000 7f4497e00640 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abac9d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e10 tx=0x7f4488011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b532f4f4f416f4e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.548+0000 7f4494c00640 1 -- 172.21.15.27:0/2077865467 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f44900e7f30 con 0x7f449810c010 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.548+0000 7f448d200640 1 -- 172.21.15.27:0/2262754456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448801e070 con 0x56299a7dd0a0 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.548+0000 7f4497400640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f447c0b65f0 0x7f447c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.548+0000 7f448d200640 1 -- 172.21.15.27:0/2262754456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4488019ae0 con 0x56299a7dd0a0 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.548+0000 7f448d200640 1 -- 172.21.15.27:0/2262754456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448800fba0 con 0x56299a7dd0a0 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 -- 172.21.15.27:0/2262754456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 msgr2=0x56299abac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abac9d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e10 tx=0x7f4488011810 comp rx=0 tx=0).stop 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f4497400640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f447c0b65f0 0x7f447c0b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f448402aa00 tx=0x7f4484041040 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 -- 172.21.15.27:0/2262754456 shutdown_connections 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7e5280 0x56299abacf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abac9d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 --2- 172.21.15.27:0/2262754456 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 -- 172.21.15.27:0/2262754456 >> 172.21.15.27:0/2262754456 conn(0x56299a7fa350 msgr2=0x56299abaa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 -- 172.21.15.27:0/2262754456 shutdown_connections 2024-09-21T01:21:30.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.549+0000 7f44a5879740 1 -- 172.21.15.27:0/2262754456 wait complete. 2024-09-21T01:21:30.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 -- start start 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 Processor -- start 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abb80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a805f60 0x56299abc3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56299a9745f0 con 0x56299a7dd0a0 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56299aaae460 con 0x56299a7e5280 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.550+0000 7f44a5879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56299abbac70 con 0x56299a805f60 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f44a5879740 10 client.0 _ll_register_callbacks cb 0x56299aabe210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abb80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abb80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47344/0 (socket says 172.21.15.27:47344) 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 -- 172.21.15.27:0/3092478044 learned_addr learned my addr 172.21.15.27:0/3092478044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f449cc00640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a805f60 0x56299abc3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 -- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a805f60 msgr2=0x56299abc3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a805f60 0x56299abc3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 -- 172.21.15.27:0/3092478044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 msgr2=0x56299abb8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497e00640 1 -- 172.21.15.27:0/3092478044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299a871540 con 0x56299a7dd0a0 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.551+0000 7f4497400640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f4497e00640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abb80d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4488009810 tx=0x7f448800fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ee463544cf0ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f4481600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f4481600640 1 -- 172.21.15.27:0/3092478044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448801e040 con 0x56299a7dd0a0 2024-09-21T01:21:30.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56299aab0a70 con 0x56299a7dd0a0 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f4481600640 1 -- 172.21.15.27:0/3092478044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4488005230 con 0x56299a7dd0a0 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f4481600640 1 -- 172.21.15.27:0/3092478044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4488030050 con 0x56299a7dd0a0 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f44a5879740 10 client.15198 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.552+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56299aacb6f0 con 0x56299a7dd0a0 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f4481600640 1 -- 172.21.15.27:0/3092478044 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56299aab0a70 con 0x56299a7dd0a0 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f4481600640 1 -- 172.21.15.27:0/3092478044 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4488019430 con 0x56299a7dd0a0 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f44a5879740 10 client.15198 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f44a5879740 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f44a5879740 20 client.15198 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f44a5879740 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a7c48e0 0x56299abca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f44a5879740 4 client.15198 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:21:30.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.553+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 --> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56299ab25ca0 con 0x56299a7c48e0 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.554+0000 7f4497400640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a7c48e0 0x56299abca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.554+0000 7f4497400640 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a7c48e0 0x56299abca380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=adb0dcd5b623d591 server_cookie=e9238bcb94d34c9a in_seq=0 out_seq=0 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.554+0000 7f4481600640 10 client.15198 ms_handle_connect on v2:172.21.15.98:6838/3799755243 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f4481600640 1 -- 172.21.15.27:0/3092478044 <== mds.3 v2:172.21.15.98:6838/3799755243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56299ab25ca0 con 0x56299a7c48e0 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f4481600640 10 client.15198 handle_command_reply: tid=0 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 client.15198 shutdown 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 msgr2=0x56299abb80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abb80d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4488009810 tx=0x7f448800fd70 comp rx=0 tx=0).stop 2024-09-21T01:21:30.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 shutdown_connections 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a7c48e0 0x56299abca380 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a805f60 0x56299abc3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 --2- 172.21.15.27:0/3092478044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7dd0a0 0x56299abb80d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 >> 172.21.15.27:0/3092478044 conn(0x56299a7fa350 msgr2=0x56299abaf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.555+0000 7f4481600640 0 client.15198 ms_handle_reset on v2:172.21.15.98:6838/3799755243 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.556+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 shutdown_connections 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.556+0000 7f44a5879740 1 -- 172.21.15.27:0/3092478044 wait complete. 2024-09-21T01:21:30.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.556+0000 7f44a5879740 20 client.15198 trim_cache size 0 max 16384 2024-09-21T01:21:30.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.575+0000 7f44a5879740 1 Processor -- start 2024-09-21T01:21:30.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.575+0000 7f44a5879740 1 -- start start 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.575+0000 7f44a5879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abb6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f44a5879740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f44a5879740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a805f60 0x56299abb80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f44a5879740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56299abc86c0 con 0x56299a805f60 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f44a5879740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56299abc8d90 con 0x56299a7e5280 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f44a5879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56299abc9460 con 0x56299a7dd0a0 2024-09-21T01:21:30.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abb6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57566/0 (socket says 172.21.15.27:57566) 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 -- 172.21.15.27:0/2709336073 learned_addr learned my addr 172.21.15.27:0/2709336073 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f449cc00640 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a805f60 0x56299abb80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 -- 172.21.15.27:0/2709336073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 msgr2=0x56299abb6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abb6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 -- 172.21.15.27:0/2709336073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a805f60 msgr2=0x56299abb80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a805f60 0x56299abb80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f4497400640 1 -- 172.21.15.27:0/2709336073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299aba45a0 con 0x56299a7e5280 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.576+0000 7f449cc00640 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a805f60 0x56299abb80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:21:30.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.577+0000 7f4497400640 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb6c90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4484031440 tx=0x7f4484009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5ebe6584d1a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:30.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.577+0000 7f4483e00640 1 -- 172.21.15.27:0/2709336073 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4484046070 con 0x56299a7e5280 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.577+0000 7f4483e00640 1 -- 172.21.15.27:0/2709336073 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4484059020 con 0x56299a7e5280 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.577+0000 7f4483e00640 1 -- 172.21.15.27:0/2709336073 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44840563f0 con 0x56299a7e5280 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 -- 172.21.15.27:0/2709336073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 msgr2=0x56299abb6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb6c90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4484031440 tx=0x7f4484009260 comp rx=0 tx=0).stop 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 -- 172.21.15.27:0/2709336073 shutdown_connections 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a805f60 0x56299abb80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a7e5280 0x56299abb6c90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 --2- 172.21.15.27:0/2709336073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abb6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 -- 172.21.15.27:0/2709336073 >> 172.21.15.27:0/2709336073 conn(0x56299a7fa350 msgr2=0x56299abb3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 -- 172.21.15.27:0/2709336073 shutdown_connections 2024-09-21T01:21:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 -- 172.21.15.27:0/2709336073 wait complete. 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.578+0000 7f44a5879740 1 -- start start 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 Processor -- start 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abcb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 0x56299abcbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abd0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56299abca200 con 0x56299a7e5280 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56299abc9b30 con 0x56299a805f60 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56299abca8d0 con 0x56299a7dd0a0 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f44a5879740 10 client.0 _ll_register_callbacks cb 0x7f4488009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f4497400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 0x56299abcbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.579+0000 7f4497400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 0x56299abcbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47362/0 (socket says 172.21.15.27:47362) 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497400640 1 -- 172.21.15.27:0/2280912231 learned_addr learned my addr 172.21.15.27:0/2280912231 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f449cc00640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abd0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abcb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 -- 172.21.15.27:0/2280912231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 msgr2=0x56299abd0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abd0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 -- 172.21.15.27:0/2280912231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 msgr2=0x56299abcbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 0x56299abcbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 -- 172.21.15.27:0/2280912231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299abd14a0 con 0x56299a7dd0a0 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f449cc00640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abd0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497400640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 0x56299abcbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4497e00640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abcb830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f44880199f0 tx=0x7f4488011e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=424e6a45cb0d9333 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.580+0000 7f4481600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f4481600640 1 -- 172.21.15.27:0/2280912231 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448803b060 con 0x56299a7dd0a0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4484059020 con 0x56299a7dd0a0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f4481600640 1 -- 172.21.15.27:0/2280912231 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f448800f5c0 con 0x56299a7dd0a0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f4481600640 1 -- 172.21.15.27:0/2280912231 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4488042050 con 0x56299a7dd0a0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f44a5879740 10 client.25234 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56299abd2860 con 0x56299a7dd0a0 2024-09-21T01:21:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.581+0000 7f4481600640 1 -- 172.21.15.27:0/2280912231 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4484059020 con 0x56299a7dd0a0 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f4481600640 1 -- 172.21.15.27:0/2280912231 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4488005490 con 0x56299a7dd0a0 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f44a5879740 10 client.25234 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f44a5879740 10 client.25234 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f44a5879740 20 client.25234 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f44a5879740 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a799c90 0x56299ac383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f44a5879740 4 client.25234 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 --> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] -- command(tid 0: {"prefix": "status"}) -- 0x56299ab21750 con 0x56299a799c90 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.582+0000 7f4497400640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a799c90 0x56299ac383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f4497400640 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a799c90 0x56299ac383d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60e154bf4f853d5 server_cookie=43a224a985a2a178 in_seq=0 out_seq=0 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f4481600640 10 client.25234 ms_handle_connect on v2:172.21.15.98:6838/3799755243 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f4481600640 1 -- 172.21.15.27:0/2280912231 <== mds.3 v2:172.21.15.98:6838/3799755243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56299ab21750 con 0x56299a799c90 2024-09-21T01:21:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f4481600640 10 client.25234 handle_command_reply: tid=0 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 client.25234 shutdown 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 msgr2=0x56299abcb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abcb830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f44880199f0 tx=0x7f4488011e40 comp rx=0 tx=0).stop 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 shutdown_connections 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x56299a799c90 0x56299ac383d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56299a805f60 0x56299abd0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56299a7e5280 0x56299abcbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f4481600640 0 client.25234 ms_handle_reset on v2:172.21.15.98:6838/3799755243 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 --2- 172.21.15.27:0/2280912231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56299a7dd0a0 0x56299abcb830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 >> 172.21.15.27:0/2280912231 conn(0x56299a7fa350 msgr2=0x56299abb6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 shutdown_connections 2024-09-21T01:21:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.583+0000 7f44a5879740 1 -- 172.21.15.27:0/2280912231 wait complete. 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f44a5879740 20 client.25234 trim_cache size 0 max 16384 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24730, 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:21:30.589 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 87.214255316999996, 2024-09-21T01:21:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:21:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:21:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:21:30.590 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 167.25369215200001 2024-09-21T01:21:30.590 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f447c0b65f0 msgr2=0x7f447c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f447c0b65f0 0x7f447c0b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f448402aa00 tx=0x7f4484041040 comp rx=0 tx=0).stop 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 msgr2=0x7f44981a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 0x7f44981a1f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f449000ec50 tx=0x7f449000c6a0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 shutdown_connections 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f447c0b65f0 0x7f447c0b8b00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.584+0000 7f449ec00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f449810c010 0x7f44981a1f30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.585+0000 7f449ec00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f449810b6c0 0x7f449819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.585+0000 7f449ec00640 1 --2- 172.21.15.27:0/2077865467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4498105d80 0x7f449819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.585+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 >> 172.21.15.27:0/2077865467 conn(0x7f44981017d0 msgr2=0x7f4498103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.585+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 shutdown_connections 2024-09-21T01:21:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:30.585+0000 7f449ec00640 1 -- 172.21.15.27:0/2077865467 wait complete. 2024-09-21T01:21:31.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T01:21:31.773 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:32 smithi027 ceph-mon[30375]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:32.302 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:32 smithi089 ceph-mon[34486]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:32 smithi098 ceph-mon[34372]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.449+0000 7ffb8cc00640 1 -- 172.21.15.27:0/446883555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810b6d0 msgr2=0x7ffb8810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.449+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/446883555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810b6d0 0x7ffb8810bae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009900 tx=0x7ffb7402a410 comp rx=0 tx=0).stop 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 -- 172.21.15.27:0/446883555 shutdown_connections 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/446883555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb8810c020 0x7ffb88112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/446883555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810b6d0 0x7ffb8810bae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/446883555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb88105d90 0x7ffb881061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 -- 172.21.15.27:0/446883555 >> 172.21.15.27:0/446883555 conn(0x7ffb88101860 msgr2=0x7ffb88103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 -- 172.21.15.27:0/446883555 shutdown_connections 2024-09-21T01:21:32.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.450+0000 7ffb8cc00640 1 -- 172.21.15.27:0/446883555 wait complete. 2024-09-21T01:21:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.451+0000 7ffb8cc00640 1 Processor -- start 2024-09-21T01:21:32.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.451+0000 7ffb8cc00640 1 -- start start 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb8cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb88105d90 0x7ffb8819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb8cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 0x7ffb8819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb8cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810c020 0x7ffb881a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb8cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88115ce0 con 0x7ffb8810c020 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb8cc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88115b60 con 0x7ffb88105d90 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb8cc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb88115e60 con 0x7ffb8810b6d0 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 0x7ffb8819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb85400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 0x7ffb8819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:48882/0 (socket says 172.21.15.27:48882) 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb85400640 1 -- 172.21.15.27:0/7266299 learned_addr learned my addr 172.21.15.27:0/7266299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb86800640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810c020 0x7ffb881a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.452+0000 7ffb85e00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb88105d90 0x7ffb8819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85400640 1 -- 172.21.15.27:0/7266299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb88105d90 msgr2=0x7ffb8819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85400640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb88105d90 0x7ffb8819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85400640 1 -- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810c020 msgr2=0x7ffb881a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85400640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810c020 0x7ffb881a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85400640 1 -- 172.21.15.27:0/7266299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb881a2640 con 0x7ffb8810b6d0 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85e00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb88105d90 0x7ffb8819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb86800640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810c020 0x7ffb881a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb85400640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 0x7ffb8819cc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb7402a930 tx=0x7ffb74031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb7ea00640 1 -- 172.21.15.27:0/7266299 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7403f030 con 0x7ffb8810b6d0 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb7ea00640 1 -- 172.21.15.27:0/7266299 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb74004490 con 0x7ffb8810b6d0 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb881a28d0 con 0x7ffb8810b6d0 2024-09-21T01:21:32.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.453+0000 7ffb7ea00640 1 -- 172.21.15.27:0/7266299 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb74035070 con 0x7ffb8810b6d0 2024-09-21T01:21:32.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.454+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb881a9090 con 0x7ffb8810b6d0 2024-09-21T01:21:32.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.457+0000 7ffb937ee740 1 Processor -- start 2024-09-21T01:21:32.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.457+0000 7ffb937ee740 1 -- start start 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b69f60 0x55fc32f10490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f10f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb937ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc32e87d50 con 0x55fc32b69f60 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb937ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fc32e77180 con 0x55fc32b410a0 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb937ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc32e7af90 con 0x55fc32b49280 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb7ea00640 1 -- 172.21.15.27:0/7266299 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7ffb74042030 con 0x7ffb8810b6d0 2024-09-21T01:21:32.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f109d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57592/0 (socket says 172.21.15.27:57592) 2024-09-21T01:21:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb85e00640 1 -- 172.21.15.27:0/2686651355 learned_addr learned my addr 172.21.15.27:0/2686651355 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.458+0000 7ffb85400640 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f10f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb86800640 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b69f60 0x55fc32f10490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb85e00640 1 -- 172.21.15.27:0/2686651355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 msgr2=0x55fc32f10f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f10f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb85e00640 1 -- 172.21.15.27:0/2686651355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b69f60 msgr2=0x55fc32f10490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b69f60 0x55fc32f10490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb85e00640 1 -- 172.21.15.27:0/2686651355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc32e41a70 con 0x55fc32b410a0 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.459+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f109d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb780115f0 tx=0x7ffb78011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105577faf51a71f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.460+0000 7ffb73000640 1 -- 172.21.15.27:0/2686651355 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7801e070 con 0x55fc32b410a0 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.460+0000 7ffb73000640 1 -- 172.21.15.27:0/2686651355 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb7800faf0 con 0x55fc32b410a0 2024-09-21T01:21:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.460+0000 7ffb73000640 1 -- 172.21.15.27:0/2686651355 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78019b60 con 0x55fc32b410a0 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb7ea00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffb6c0b6670 0x7ffb6c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 -- 172.21.15.27:0/2686651355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 msgr2=0x55fc32f109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f109d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb780115f0 tx=0x7ffb78011ab0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 -- 172.21.15.27:0/2686651355 shutdown_connections 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f10f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f109d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2686651355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b69f60 0x55fc32f10490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 -- 172.21.15.27:0/2686651355 >> 172.21.15.27:0/2686651355 conn(0x55fc32b5e350 msgr2=0x55fc32f0e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:32.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb7ea00640 1 -- 172.21.15.27:0/7266299 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ffb74107930 con 0x7ffb8810b6d0 2024-09-21T01:21:32.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb86800640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffb6c0b6670 0x7ffb6c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.461+0000 7ffb937ee740 1 -- 172.21.15.27:0/2686651355 shutdown_connections 2024-09-21T01:21:32.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.462+0000 7ffb937ee740 1 -- 172.21.15.27:0/2686651355 wait complete. 2024-09-21T01:21:32.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.462+0000 7ffb86800640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffb6c0b6670 0x7ffb6c0b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005fd0 tx=0x7ffb80009500 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:32.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.462+0000 7ffb937ee740 1 -- start start 2024-09-21T01:21:32.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.462+0000 7ffb937ee740 1 Processor -- start 2024-09-21T01:21:32.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b69f60 0x55fc32f27f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc32cd85f0 con 0x55fc32b49280 2024-09-21T01:21:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fc32e12460 con 0x55fc32b410a0 2024-09-21T01:21:32.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f1ec70 con 0x55fc32b69f60 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb937ee740 10 client.0 _ll_register_callbacks cb 0x55fc32e22210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f1c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57594/0 (socket says 172.21.15.27:57594) 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb85e00640 1 -- 172.21.15.27:0/3121757534 learned_addr learned my addr 172.21.15.27:0/3121757534 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb85400640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.463+0000 7ffb86800640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b69f60 0x55fc32f27f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb85e00640 1 -- 172.21.15.27:0/3121757534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b69f60 msgr2=0x55fc32f27f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb85e00640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b69f60 0x55fc32f27f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb85e00640 1 -- 172.21.15.27:0/3121757534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 msgr2=0x55fc32f1c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb85e00640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f1c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb85e00640 1 -- 172.21.15.27:0/3121757534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc32bd5540 con 0x55fc32b410a0 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb86800640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b69f60 0x55fc32f27f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:21:32.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb85e00640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f1c080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffb7801c040 tx=0x7ffb780056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06a79a0db080239 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb67400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb67400640 1 -- 172.21.15.27:0/3121757534 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7801e040 con 0x55fc32b410a0 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc32e14a70 con 0x55fc32b410a0 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.464+0000 7ffb67400640 1 -- 172.21.15.27:0/3121757534 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ffb780194d0 con 0x55fc32b410a0 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.465+0000 7ffb67400640 1 -- 172.21.15.27:0/3121757534 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb78030050 con 0x55fc32b410a0 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.465+0000 7ffb937ee740 10 client.25256 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.465+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc32e2f6f0 con 0x55fc32b410a0 2024-09-21T01:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.465+0000 7ffb67400640 1 -- 172.21.15.27:0/3121757534 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fc32e14a70 con 0x55fc32b410a0 2024-09-21T01:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.465+0000 7ffb67400640 1 -- 172.21.15.27:0/3121757534 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffb78021070 con 0x55fc32b410a0 2024-09-21T01:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb937ee740 10 client.25256 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb937ee740 10 client.25256 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb937ee740 20 client.25256 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb937ee740 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b288e0 0x55fc32f2e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb937ee740 4 client.25256 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 --> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc32e89ca0 con 0x55fc32b288e0 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.466+0000 7ffb85400640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b288e0 0x55fc32f2e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.467+0000 7ffb85400640 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b288e0 0x55fc32f2e270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=352da392b9e63b57 server_cookie=3b1e425d92b5c777 in_seq=0 out_seq=0 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.467+0000 7ffb67400640 10 client.25256 ms_handle_connect on v2:172.21.15.89:6836/3556269004 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb67400640 1 -- 172.21.15.27:0/3121757534 <== mds.2 v2:172.21.15.89:6836/3556269004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc32e89ca0 con 0x55fc32b288e0 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb67400640 10 client.25256 handle_command_reply: tid=0 2024-09-21T01:21:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 client.25256 shutdown 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 msgr2=0x55fc32f1c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f1c080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffb7801c040 tx=0x7ffb780056d0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 shutdown_connections 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b288e0 0x55fc32f2e270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b69f60 0x55fc32f27f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f1c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 --2- 172.21.15.27:0/3121757534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f1c080 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb67400640 0 client.25256 ms_handle_reset on v2:172.21.15.89:6836/3556269004 2024-09-21T01:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 >> 172.21.15.27:0/3121757534 conn(0x55fc32b5e350 msgr2=0x55fc32f13670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 shutdown_connections 2024-09-21T01:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.468+0000 7ffb937ee740 1 -- 172.21.15.27:0/3121757534 wait complete. 2024-09-21T01:21:32.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.469+0000 7ffb937ee740 20 client.25256 trim_cache size 0 max 16384 2024-09-21T01:21:32.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 Processor -- start 2024-09-21T01:21:32.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 -- start start 2024-09-21T01:21:32.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b5e350 0x55fc32f2ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b410a0 0x55fc32f27ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f15710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f2c5b0 con 0x55fc32b49280 2024-09-21T01:21:32.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f2cc80 con 0x55fc32b5e350 2024-09-21T01:21:32.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.488+0000 7ffb937ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f2d350 con 0x55fc32b410a0 2024-09-21T01:21:32.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f15710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f15710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47404/0 (socket says 172.21.15.27:47404) 2024-09-21T01:21:32.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 -- 172.21.15.27:0/2107076288 learned_addr learned my addr 172.21.15.27:0/2107076288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:32.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b5e350 0x55fc32f2ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb85400640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b410a0 0x55fc32f27ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 -- 172.21.15.27:0/2107076288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b410a0 msgr2=0x55fc32f27ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b410a0 0x55fc32f27ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 -- 172.21.15.27:0/2107076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b5e350 msgr2=0x55fc32f2ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b5e350 0x55fc32f2ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb86800640 1 -- 172.21.15.27:0/2107076288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc32f1a380 con 0x55fc32b49280 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b5e350 0x55fc32f2ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.489+0000 7ffb85400640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b410a0 0x55fc32f27ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.490+0000 7ffb86800640 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f15710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb80012600 tx=0x7ffb80014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5f2d7bc5cc4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.490+0000 7ffb72600640 1 -- 172.21.15.27:0/2107076288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb800206f0 con 0x55fc32b49280 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.490+0000 7ffb72600640 1 -- 172.21.15.27:0/2107076288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb80024070 con 0x55fc32b49280 2024-09-21T01:21:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.490+0000 7ffb72600640 1 -- 172.21.15.27:0/2107076288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8001fcb0 con 0x55fc32b49280 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 -- 172.21.15.27:0/2107076288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 msgr2=0x55fc32f15710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f15710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffb80012600 tx=0x7ffb80014000 comp rx=0 tx=0).stop 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 -- 172.21.15.27:0/2107076288 shutdown_connections 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b49280 0x55fc32f15710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b410a0 0x55fc32f27ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2107076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b5e350 0x55fc32f2ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 -- 172.21.15.27:0/2107076288 >> 172.21.15.27:0/2107076288 conn(0x55fc32f2c1a0 msgr2=0x55fc32f17030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 -- 172.21.15.27:0/2107076288 shutdown_connections 2024-09-21T01:21:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.491+0000 7ffb937ee740 1 -- 172.21.15.27:0/2107076288 wait complete. 2024-09-21T01:21:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.492+0000 7ffb937ee740 1 -- start start 2024-09-21T01:21:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.492+0000 7ffb937ee740 1 Processor -- start 2024-09-21T01:21:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f16780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f36530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b5e350 0x55fc32f36c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f2e0f0 con 0x55fc32b5e350 2024-09-21T01:21:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f2da20 con 0x55fc32b410a0 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fc32f2e7c0 con 0x55fc32b49280 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb937ee740 10 client.0 _ll_register_callbacks cb 0x7ffb78011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb86800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b5e350 0x55fc32f36c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f16780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb85e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f16780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57620/0 (socket says 172.21.15.27:57620) 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb85e00640 1 -- 172.21.15.27:0/2468907820 learned_addr learned my addr 172.21.15.27:0/2468907820 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.493+0000 7ffb85400640 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f36530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb85e00640 1 -- 172.21.15.27:0/2468907820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 msgr2=0x55fc32f36530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f36530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb85e00640 1 -- 172.21.15.27:0/2468907820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b5e350 msgr2=0x55fc32f36c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b5e350 0x55fc32f36c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb85e00640 1 -- 172.21.15.27:0/2468907820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc32f2f470 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb85e00640 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f16780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffb7801c040 tx=0x7ffb78009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d37372b33329b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb67400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.494+0000 7ffb67400640 1 -- 172.21.15.27:0/2468907820 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78039060 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb67400640 1 -- 172.21.15.27:0/2468907820 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ffb7800f840 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb80024070 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb67400640 1 -- 172.21.15.27:0/2468907820 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb78040050 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb937ee740 10 client.25262 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc32f99910 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb67400640 1 -- 172.21.15.27:0/2468907820 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb80024070 con 0x55fc32b410a0 2024-09-21T01:21:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb67400640 1 -- 172.21.15.27:0/2468907820 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffb78021040 con 0x55fc32b410a0 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.495+0000 7ffb937ee740 10 client.25262 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb937ee740 10 client.25262 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb937ee740 20 client.25262 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b69f60 0x55fc32f9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb937ee740 4 client.25262 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 --> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] -- command(tid 0: {"prefix": "status"}) -- 0x55fc32e85750 con 0x55fc32b69f60 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb85400640 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b69f60 0x55fc32f9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb85400640 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b69f60 0x55fc32f9d110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d95ade7c5437411 server_cookie=ff76c6f1a0273022 in_seq=0 out_seq=0 2024-09-21T01:21:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.496+0000 7ffb67400640 10 client.25262 ms_handle_connect on v2:172.21.15.89:6836/3556269004 2024-09-21T01:21:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb67400640 1 -- 172.21.15.27:0/2468907820 <== mds.2 v2:172.21.15.89:6836/3556269004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fc32e85750 con 0x55fc32b69f60 2024-09-21T01:21:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb67400640 10 client.25262 handle_command_reply: tid=0 2024-09-21T01:21:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 client.25262 shutdown 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 msgr2=0x55fc32f16780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f16780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffb7801c040 tx=0x7ffb78009f60 comp rx=0 tx=0).stop 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 shutdown_connections 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55fc32b69f60 0x55fc32f9d110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55fc32b5e350 0x55fc32f36c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fc32b49280 0x55fc32f36530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 --2- 172.21.15.27:0/2468907820 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55fc32b410a0 0x55fc32f16780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb67400640 0 client.25262 ms_handle_reset on v2:172.21.15.89:6836/3556269004 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.497+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 >> 172.21.15.27:0/2468907820 conn(0x55fc32f2c1a0 msgr2=0x55fc32f1a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 shutdown_connections 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb937ee740 1 -- 172.21.15.27:0/2468907820 wait complete. 2024-09-21T01:21:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb937ee740 20 client.25262 trim_cache size 0 max 16384 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24734, 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 90.139410143999996, 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:21:32.501 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:21:32.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 171.81573799099999 2024-09-21T01:21:32.502 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:21:32.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffb6c0b6670 msgr2=0x7ffb6c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffb6c0b6670 0x7ffb6c0b8b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005fd0 tx=0x7ffb80009500 comp rx=0 tx=0).stop 2024-09-21T01:21:32.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 msgr2=0x7ffb8819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:32.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.498+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 0x7ffb8819cc70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb7402a930 tx=0x7ffb74031a20 comp rx=0 tx=0).stop 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 shutdown_connections 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ffb6c0b6670 0x7ffb6c0b8b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb8810c020 0x7ffb881a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb8810b6d0 0x7ffb8819cc70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 --2- 172.21.15.27:0/7266299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffb88105d90 0x7ffb8819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 >> 172.21.15.27:0/7266299 conn(0x7ffb88101860 msgr2=0x7ffb88103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 shutdown_connections 2024-09-21T01:21:32.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:32.499+0000 7ffb8cc00640 1 -- 172.21.15.27:0/7266299 wait complete. 2024-09-21T01:21:33.317 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.g status 2024-09-21T01:21:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:34.069 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:34 smithi027 ceph-mon[30375]: pgmap v374: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 -- 172.21.15.27:0/2634358324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 msgr2=0x7f39fc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 --2- 172.21.15.27:0/2634358324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc110470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009ca0 tx=0x7f39f802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 -- 172.21.15.27:0/2634358324 shutdown_connections 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 --2- 172.21.15.27:0/2634358324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc110470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 --2- 172.21.15.27:0/2634358324 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1092f0 0x7f39fc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 --2- 172.21.15.27:0/2634358324 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1039b0 0x7f39fc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 -- 172.21.15.27:0/2634358324 >> 172.21.15.27:0/2634358324 conn(0x7f39fc0ff440 msgr2=0x7f39fc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 -- 172.21.15.27:0/2634358324 shutdown_connections 2024-09-21T01:21:34.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.382+0000 7f3a03200640 1 -- 172.21.15.27:0/2634358324 wait complete. 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.383+0000 7f3a03200640 1 Processor -- start 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.383+0000 7f3a03200640 1 -- start start 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a03200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1039b0 0x7f39fc077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a03200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1092f0 0x7f39fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a03200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a03200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc113950 con 0x7f39fc109c40 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a03200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc1137d0 con 0x7f39fc1092f0 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a03200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc113ad0 con 0x7f39fc1039b0 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a01e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1039b0 0x7f39fc077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a01400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1092f0 0x7f39fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47434/0 (socket says 172.21.15.27:47434) 2024-09-21T01:21:34.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a02800640 1 -- 172.21.15.27:0/4019078333 learned_addr learned my addr 172.21.15.27:0/4019078333 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:34.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a01e00640 1 -- 172.21.15.27:0/4019078333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1092f0 msgr2=0x7f39fc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a01e00640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1092f0 0x7f39fc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a01e00640 1 -- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 msgr2=0x7f39fc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.384+0000 7f3a01e00640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f3a01e00640 1 -- 172.21.15.27:0/4019078333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc076470 con 0x7f39fc1039b0 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f3a01400640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1092f0 0x7f39fc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f3a01e00640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1039b0 0x7f39fc077120 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39f000e9f0 tx=0x7f39f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f39f6a00640 1 -- 172.21.15.27:0/4019078333 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f000ccc0 con 0x7f39fc1039b0 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f39f6a00640 1 -- 172.21.15.27:0/4019078333 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f00108e0 con 0x7f39fc1039b0 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc1acc80 con 0x7f39fc1039b0 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f39f6a00640 1 -- 172.21.15.27:0/4019078333 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0018880 con 0x7f39fc1039b0 2024-09-21T01:21:34.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.385+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc1ad1c0 con 0x7f39fc1039b0 2024-09-21T01:21:34.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.388+0000 7f3a0b421740 1 Processor -- start 2024-09-21T01:21:34.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 -- start start 2024-09-21T01:21:34.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08d3f60 0x5564f0c7a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08ab0a0 0x5564f0c7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564f0bf1d50 con 0x5564f08b3280 2024-09-21T01:21:34.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564f0be1180 con 0x5564f08d3f60 2024-09-21T01:21:34.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a0b421740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5564f0be4f90 con 0x5564f08ab0a0 2024-09-21T01:21:34.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f39f6a00640 1 -- 172.21.15.27:0/4019078333 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f39f0010430 con 0x7f39fc1039b0 2024-09-21T01:21:34.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c7b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47450/0 (socket says 172.21.15.27:47450) 2024-09-21T01:21:34.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a02800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08d3f60 0x5564f0c7a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.389+0000 7f3a01400640 1 -- 172.21.15.27:0/221092901 learned_addr learned my addr 172.21.15.27:0/221092901 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:34.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a01e00640 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08ab0a0 0x5564f0c7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a02800640 1 -- 172.21.15.27:0/221092901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08ab0a0 msgr2=0x5564f0c7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a02800640 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08ab0a0 0x5564f0c7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a02800640 1 -- 172.21.15.27:0/221092901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 msgr2=0x5564f0c7b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a02800640 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c7b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a02800640 1 -- 172.21.15.27:0/221092901 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f0baba70 con 0x5564f08d3f60 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.390+0000 7f3a02800640 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08d3f60 0x5564f0c7a670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004af0 tx=0x7f39f8009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5378021d09c6cf99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.391+0000 7f39eb000640 1 -- 172.21.15.27:0/221092901 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8031cf0 con 0x5564f08d3f60 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.391+0000 7f39eb000640 1 -- 172.21.15.27:0/221092901 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f8036030 con 0x5564f08d3f60 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.391+0000 7f39eb000640 1 -- 172.21.15.27:0/221092901 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80042a0 con 0x5564f08d3f60 2024-09-21T01:21:34.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.391+0000 7f39f6a00640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39e40b6670 0x7f39e40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 -- 172.21.15.27:0/221092901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08d3f60 msgr2=0x5564f0c7a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08d3f60 0x5564f0c7a670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004af0 tx=0x7f39f8009450 comp rx=0 tx=0).stop 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f39f6a00640 1 -- 172.21.15.27:0/4019078333 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f39f00e7cb0 con 0x7f39fc1039b0 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a01e00640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39e40b6670 0x7f39e40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 -- 172.21.15.27:0/221092901 shutdown_connections 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08ab0a0 0x5564f0c7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 --2- 172.21.15.27:0/221092901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08d3f60 0x5564f0c7a670 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 -- 172.21.15.27:0/221092901 >> 172.21.15.27:0/221092901 conn(0x5564f08c8350 msgr2=0x5564f0c78560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 -- 172.21.15.27:0/221092901 shutdown_connections 2024-09-21T01:21:34.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.392+0000 7f3a0b421740 1 -- 172.21.15.27:0/221092901 wait complete. 2024-09-21T01:21:34.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.393+0000 7f3a01e00640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39e40b6670 0x7f39e40b8b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39f00e51a0 tx=0x7f39f0009700 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:34.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.393+0000 7f3a0b421740 1 -- start start 2024-09-21T01:21:34.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.393+0000 7f3a0b421740 1 Processor -- start 2024-09-21T01:21:34.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 0x5564f0c86220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564f0a425f0 con 0x5564f08b3280 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564f0b7c460 con 0x5564f08ab0a0 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c88c70 con 0x5564f08d3f60 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a0b421740 10 client.0 _ll_register_callbacks cb 0x5564f0b8c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a02800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 0x5564f0c86220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a01400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a02800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 0x5564f0c86220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57650/0 (socket says 172.21.15.27:57650) 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a02800640 1 -- 172.21.15.27:0/4116968773 learned_addr learned my addr 172.21.15.27:0/4116968773 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:34.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.394+0000 7f3a02800640 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c91f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a02800640 1 -- 172.21.15.27:0/4116968773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 msgr2=0x5564f0c91f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a02800640 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c91f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a02800640 1 -- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 msgr2=0x5564f0c86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a02800640 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a02800640 1 -- 172.21.15.27:0/4116968773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f093f540 con 0x5564f08ab0a0 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a02800640 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 0x5564f0c86220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39f8003e20 tx=0x7f39f8003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e30575756ac5c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f39df400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f39df400640 1 -- 172.21.15.27:0/4116968773 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8058060 con 0x5564f08ab0a0 2024-09-21T01:21:34.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564f0b7ea70 con 0x5564f08ab0a0 2024-09-21T01:21:34.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.395+0000 7f39df400640 1 -- 172.21.15.27:0/4116968773 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f39f8004680 con 0x5564f08ab0a0 2024-09-21T01:21:34.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f39df400640 1 -- 172.21.15.27:0/4116968773 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39f804a040 con 0x5564f08ab0a0 2024-09-21T01:21:34.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f3a0b421740 10 client.25274 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:34.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564f0b996f0 con 0x5564f08ab0a0 2024-09-21T01:21:34.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f39df400640 1 -- 172.21.15.27:0/4116968773 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5564f0b7ea70 con 0x5564f08ab0a0 2024-09-21T01:21:34.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f39df400640 1 -- 172.21.15.27:0/4116968773 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f39f8050070 con 0x5564f08ab0a0 2024-09-21T01:21:34.400 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:34 smithi098 ceph-mon[34372]: pgmap v374: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f3a0b421740 10 client.25274 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f3a0b421740 10 client.25274 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.396+0000 7f3a0b421740 20 client.25274 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.397+0000 7f3a0b421740 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f08928e0 0x5564f0c98180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.397+0000 7f3a0b421740 4 client.25274 mds_command: new command op to 24752 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.397+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 --> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564f0bf3ca0 con 0x5564f08928e0 2024-09-21T01:21:34.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.397+0000 7f3a01400640 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f08928e0 0x5564f0c98180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.397+0000 7f3a01400640 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f08928e0 0x5564f0c98180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6227f93dd41a3134 server_cookie=708bf3d8a63e01c5 in_seq=0 out_seq=0 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.397+0000 7f39df400640 10 client.25274 ms_handle_connect on v2:172.21.15.27:6838/3115497172 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.398+0000 7f39df400640 1 -- 172.21.15.27:0/4116968773 <== mds.1 v2:172.21.15.27:6838/3115497172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564f0bf3ca0 con 0x5564f08928e0 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.398+0000 7f39df400640 10 client.25274 handle_command_reply: tid=0 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.398+0000 7f3a0b421740 1 client.25274 shutdown 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 msgr2=0x5564f0c86220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 0x5564f0c86220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39f8003e20 tx=0x7f39f8003d20 comp rx=0 tx=0).stop 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 shutdown_connections 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f08928e0 0x5564f0c98180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c91f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08b3280 0x5564f0c86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 --2- 172.21.15.27:0/4116968773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08ab0a0 0x5564f0c86220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f39df400640 0 client.25274 ms_handle_reset on v2:172.21.15.27:6838/3115497172 2024-09-21T01:21:34.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 >> 172.21.15.27:0/4116968773 conn(0x5564f08c8350 msgr2=0x5564f0c7d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:34.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 shutdown_connections 2024-09-21T01:21:34.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 1 -- 172.21.15.27:0/4116968773 wait complete. 2024-09-21T01:21:34.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.399+0000 7f3a0b421740 20 client.25274 trim_cache size 0 max 16384 2024-09-21T01:21:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.418+0000 7f3a0b421740 1 Processor -- start 2024-09-21T01:21:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.418+0000 7f3a0b421740 1 -- start start 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0c96d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c91ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a0b421740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c964c0 con 0x5564f08ab0a0 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a0b421740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c96b90 con 0x5564f08b3280 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a0b421740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c97260 con 0x5564f08d3f60 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0c96d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0c96d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47476/0 (socket says 172.21.15.27:47476) 2024-09-21T01:21:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a02800640 1 -- 172.21.15.27:0/502906801 learned_addr learned my addr 172.21.15.27:0/502906801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.419+0000 7f3a01400640 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c91ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a02800640 1 -- 172.21.15.27:0/502906801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 msgr2=0x5564f0c80790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a02800640 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a02800640 1 -- 172.21.15.27:0/502906801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 msgr2=0x5564f0c91ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a02800640 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c91ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a02800640 1 -- 172.21.15.27:0/502906801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f0c872e0 con 0x5564f08ab0a0 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a01400640 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c91ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f3a02800640 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0c96d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39f802f540 tx=0x7f39f8004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02cb38eccd0d63d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f39ea600640 1 -- 172.21.15.27:0/502906801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8058070 con 0x5564f08ab0a0 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f39ea600640 1 -- 172.21.15.27:0/502906801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f8036070 con 0x5564f08ab0a0 2024-09-21T01:21:34.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.420+0000 7f39ea600640 1 -- 172.21.15.27:0/502906801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80046e0 con 0x5564f08ab0a0 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 -- 172.21.15.27:0/502906801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 msgr2=0x5564f0c96d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0c96d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39f802f540 tx=0x7f39f8004350 comp rx=0 tx=0).stop 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 -- 172.21.15.27:0/502906801 shutdown_connections 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c91ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 --2- 172.21.15.27:0/502906801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0c96d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 -- 172.21.15.27:0/502906801 >> 172.21.15.27:0/502906801 conn(0x5564f08c8350 msgr2=0x5564f0c81160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.421+0000 7f3a0b421740 1 -- 172.21.15.27:0/502906801 shutdown_connections 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.422+0000 7f3a0b421740 1 -- 172.21.15.27:0/502906801 wait complete. 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.422+0000 7f3a0b421740 1 -- start start 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.422+0000 7f3a0b421740 1 Processor -- start 2024-09-21T01:21:34.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0ca0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c9a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c98000 con 0x5564f08ab0a0 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c97930 con 0x5564f08b3280 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5564f0c986d0 con 0x5564f08d3f60 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a0b421740 10 client.0 _ll_register_callbacks cb 0x7f39f8003b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0ca0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a02800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0ca0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47486/0 (socket says 172.21.15.27:47486) 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a02800640 1 -- 172.21.15.27:0/3967744993 learned_addr learned my addr 172.21.15.27:0/3967744993 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a01400640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c9a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.423+0000 7f3a02800640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a02800640 1 -- 172.21.15.27:0/3967744993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 msgr2=0x5564f0c9a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a02800640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c9a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a02800640 1 -- 172.21.15.27:0/3967744993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 msgr2=0x5564f0c9a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a02800640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c9a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a02800640 1 -- 172.21.15.27:0/3967744993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f0c9ac50 con 0x5564f08ab0a0 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a02800640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0ca0b70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f39f802f510 tx=0x7f39f802f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac29d34d2b70134 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f39df400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f39df400640 1 -- 172.21.15.27:0/3967744993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8058040 con 0x5564f08ab0a0 2024-09-21T01:21:34.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f39df400640 1 -- 172.21.15.27:0/3967744993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f39f802fc90 con 0x5564f08ab0a0 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39f8036070 con 0x5564f08ab0a0 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f39df400640 1 -- 172.21.15.27:0/3967744993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39f8004b40 con 0x5564f08ab0a0 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a0b421740 10 client.15237 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.424+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564f0d03650 con 0x5564f08ab0a0 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f39df400640 1 -- 172.21.15.27:0/3967744993 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f8036070 con 0x5564f08ab0a0 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f39df400640 1 -- 172.21.15.27:0/3967744993 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f39f8050070 con 0x5564f08ab0a0 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a0b421740 10 client.15237 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a0b421740 10 client.15237 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a0b421740 20 client.15237 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a0b421740 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f0867c90 0x5564f0d068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a0b421740 4 client.15237 mds_command: new command op to 24752 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 --> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] -- command(tid 0: {"prefix": "status"}) -- 0x5564f0bef750 con 0x5564f0867c90 2024-09-21T01:21:34.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.425+0000 7f3a01400640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f0867c90 0x5564f0d068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.426+0000 7f3a01400640 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f0867c90 0x5564f0d068c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21a65357d6ba839b server_cookie=35104f97af7283ab in_seq=0 out_seq=0 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.426+0000 7f39df400640 10 client.15237 ms_handle_connect on v2:172.21.15.27:6838/3115497172 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.426+0000 7f39df400640 1 -- 172.21.15.27:0/3967744993 <== mds.1 v2:172.21.15.27:6838/3115497172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5564f0bef750 con 0x5564f0867c90 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.426+0000 7f39df400640 10 client.15237 handle_command_reply: tid=0 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.426+0000 7f3a0b421740 1 client.15237 shutdown 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 msgr2=0x5564f0ca0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0ca0b70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f39f802f510 tx=0x7f39f802f9d0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 shutdown_connections 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x5564f0867c90 0x5564f0d068c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5564f08d3f60 0x5564f0c9a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5564f08b3280 0x5564f0c9a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 --2- 172.21.15.27:0/3967744993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564f08ab0a0 0x5564f0ca0b70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 >> 172.21.15.27:0/3967744993 conn(0x5564f08c8350 msgr2=0x5564f0c77df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f39df400640 0 client.15237 ms_handle_reset on v2:172.21.15.27:6838/3115497172 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 shutdown_connections 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 1 -- 172.21.15.27:0/3967744993 wait complete. 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.427+0000 7f3a0b421740 20 client.15237 trim_cache size 0 max 16384 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T01:21:34.431 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24752, 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 93.078464803000003, 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 167.01382420499999 2024-09-21T01:21:34.432 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39e40b6670 msgr2=0x7f39e40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39e40b6670 0x7f39e40b8b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39f00e51a0 tx=0x7f39f0009700 comp rx=0 tx=0).stop 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1039b0 msgr2=0x7f39fc077120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1039b0 0x7f39fc077120 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f39f000e9f0 tx=0x7f39f000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 shutdown_connections 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f39e40b6670 0x7f39e40b8b80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39fc109c40 0x7f39fc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f39fc1092f0 0x7f39fc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 --2- 172.21.15.27:0/4019078333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39fc1039b0 0x7f39fc077120 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 >> 172.21.15.27:0/4019078333 conn(0x7f39fc0ff440 msgr2=0x7f39fc108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 shutdown_connections 2024-09-21T01:21:34.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:34.428+0000 7f3a03200640 1 -- 172.21.15.27:0/4019078333 wait complete. 2024-09-21T01:21:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:34 smithi089 ceph-mon[34486]: pgmap v374: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:35.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T01:21:35.637 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:36.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 -- 172.21.15.27:0/1839300154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c075730 msgr2=0x7f3c1c075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 --2- 172.21.15.27:0/1839300154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c075730 0x7f3c1c075b40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009900 tx=0x7f3c1002a410 comp rx=0 tx=0).stop 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 -- 172.21.15.27:0/1839300154 shutdown_connections 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 --2- 172.21.15.27:0/1839300154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 --2- 172.21.15.27:0/1839300154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c1c076080 0x7f3c1c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 --2- 172.21.15.27:0/1839300154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c075730 0x7f3c1c075b40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 -- 172.21.15.27:0/1839300154 >> 172.21.15.27:0/1839300154 conn(0x7f3c1c07b850 msgr2=0x7f3c1c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 -- 172.21.15.27:0/1839300154 shutdown_connections 2024-09-21T01:21:36.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.315+0000 7f3c24200640 1 -- 172.21.15.27:0/1839300154 wait complete. 2024-09-21T01:21:36.317 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:36 smithi027 ceph-mon[30375]: pgmap v375: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.316+0000 7f3c24200640 1 Processor -- start 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 -- start start 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c1c075730 0x7f3c1c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c076080 0x7f3c1c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c115cf0 con 0x7f3c1c076080 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c115b70 con 0x7f3c1c110140 2024-09-21T01:21:36.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c24200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c115e70 con 0x7f3c1c075730 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c22200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c22200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c1a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49034/0 (socket says 172.21.15.27:49034) 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c22200640 1 -- 172.21.15.27:0/2597871684 learned_addr learned my addr 172.21.15.27:0/2597871684 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.317+0000 7f3c20e00640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c076080 0x7f3c1c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c21800640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c1c075730 0x7f3c1c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c22200640 1 -- 172.21.15.27:0/2597871684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c1c075730 msgr2=0x7f3c1c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c22200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c1c075730 0x7f3c1c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c22200640 1 -- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c076080 msgr2=0x7f3c1c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c22200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c076080 0x7f3c1c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c22200640 1 -- 172.21.15.27:0/2597871684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c1a2610 con 0x7f3c1c110140 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c20e00640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c076080 0x7f3c1c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:36.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.318+0000 7f3c22200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c1a1ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009fd0 tx=0x7f3c1400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.319+0000 7f3c1a000640 1 -- 172.21.15.27:0/2597871684 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14017070 con 0x7f3c1c110140 2024-09-21T01:21:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.319+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c1a2900 con 0x7f3c1c110140 2024-09-21T01:21:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.319+0000 7f3c1a000640 1 -- 172.21.15.27:0/2597871684 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c14003e20 con 0x7f3c1c110140 2024-09-21T01:21:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.319+0000 7f3c1a000640 1 -- 172.21.15.27:0/2597871684 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14010460 con 0x7f3c1c110140 2024-09-21T01:21:36.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.319+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1a9060 con 0x7f3c1c110140 2024-09-21T01:21:36.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.323+0000 7f3c2ae01740 1 Processor -- start 2024-09-21T01:21:36.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.324+0000 7f3c1a000640 1 -- 172.21.15.27:0/2597871684 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3c14003970 con 0x7f3c1c110140 2024-09-21T01:21:36.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.324+0000 7f3c2ae01740 1 -- start start 2024-09-21T01:21:36.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.324+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.324+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76b99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.324+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72f2280 0x55eac76b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eac7630d50 con 0x55eac72f2280 2024-09-21T01:21:36.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eac7620180 con 0x55eac7312f60 2024-09-21T01:21:36.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eac7623f90 con 0x55eac72ea0a0 2024-09-21T01:21:36.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c21800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76b99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49048/0 (socket says 172.21.15.27:49048) 2024-09-21T01:21:36.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c20e00640 1 -- 172.21.15.27:0/3080964438 learned_addr learned my addr 172.21.15.27:0/3080964438 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c20e00640 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72f2280 0x55eac76b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c21800640 1 -- 172.21.15.27:0/3080964438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 msgr2=0x55eac76b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c21800640 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c21800640 1 -- 172.21.15.27:0/3080964438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72f2280 msgr2=0x55eac76b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c21800640 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72f2280 0x55eac76b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.325+0000 7f3c21800640 1 -- 172.21.15.27:0/3080964438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac75eaa70 con 0x55eac72ea0a0 2024-09-21T01:21:36.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c21800640 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76b99d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c1002aa00 tx=0x7f3c10004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2616c016a71c7b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c1a000640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c040b65f0 0x7f3c040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c0ea00640 1 -- 172.21.15.27:0/3080964438 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10035040 con 0x55eac72ea0a0 2024-09-21T01:21:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c1a000640 1 -- 172.21.15.27:0/2597871684 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3c140e76b0 con 0x7f3c1c110140 2024-09-21T01:21:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c0ea00640 1 -- 172.21.15.27:0/3080964438 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c100054f0 con 0x55eac72ea0a0 2024-09-21T01:21:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c0ea00640 1 -- 172.21.15.27:0/3080964438 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c100035f0 con 0x55eac72ea0a0 2024-09-21T01:21:36.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.326+0000 7f3c20e00640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c040b65f0 0x7f3c040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.327+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3080964438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 msgr2=0x55eac76b99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.327+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76b99d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c1002aa00 tx=0x7f3c10004170 comp rx=0 tx=0).stop 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.327+0000 7f3c20e00640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c040b65f0 0x7f3c040b8b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c080058c0 tx=0x7f3c08005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3080964438 shutdown_connections 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72f2280 0x55eac76b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76b99d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3080964438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3080964438 >> 172.21.15.27:0/3080964438 conn(0x55eac7307350 msgr2=0x55eac76b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:36.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3080964438 shutdown_connections 2024-09-21T01:21:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3080964438 wait complete. 2024-09-21T01:21:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 -- start start 2024-09-21T01:21:36.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.328+0000 7f3c2ae01740 1 Processor -- start 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eac74815f0 con 0x55eac7312f60 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eac75bb460 con 0x55eac72f2280 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eac76c7c10 con 0x55eac72ea0a0 2024-09-21T01:21:36.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c2ae01740 10 client.0 _ll_register_callbacks cb 0x55eac75cb210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c22200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c20e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c22200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36946/0 (socket says 172.21.15.27:36946) 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c22200640 1 -- 172.21.15.27:0/1244710533 learned_addr learned my addr 172.21.15.27:0/1244710533 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.329+0000 7f3c21800640 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c22200640 1 -- 172.21.15.27:0/1244710533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 msgr2=0x55eac76c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c22200640 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c22200640 1 -- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 msgr2=0x55eac76c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c22200640 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c22200640 1 -- 172.21.15.27:0/1244710533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac737e540 con 0x55eac7312f60 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c22200640 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76d0f00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c140ad230 tx=0x7f3c140e4ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e02bc499326b904 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c03400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c03400640 1 -- 172.21.15.27:0/1244710533 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14020040 con 0x55eac7312f60 2024-09-21T01:21:36.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c03400640 1 -- 172.21.15.27:0/1244710533 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3c140a8bf0 con 0x55eac7312f60 2024-09-21T01:21:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.330+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eac75bda70 con 0x55eac7312f60 2024-09-21T01:21:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c03400640 1 -- 172.21.15.27:0/1244710533 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c140acdb0 con 0x55eac7312f60 2024-09-21T01:21:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 10 client.15249 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:36.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eac75d86f0 con 0x55eac7312f60 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c03400640 1 -- 172.21.15.27:0/1244710533 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eac75bda70 con 0x55eac7312f60 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c03400640 1 -- 172.21.15.27:0/1244710533 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3c140b2e60 con 0x55eac7312f60 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 10 client.15249 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 10 client.15249 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 20 client.15249 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72d18e0 0x55eac76d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 4 client.15249 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.331+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eac7632ca0 con 0x55eac72d18e0 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.332+0000 7f3c21800640 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72d18e0 0x55eac76d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.333+0000 7f3c21800640 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72d18e0 0x55eac76d7600 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1356a5f2edf7a6bb server_cookie=8720479013f5b5c6 in_seq=0 out_seq=0 2024-09-21T01:21:36.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.333+0000 7f3c03400640 10 client.15249 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:21:36.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c03400640 1 -- 172.21.15.27:0/1244710533 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eac7632ca0 con 0x55eac72d18e0 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c03400640 10 client.15249 handle_command_reply: tid=0 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 client.15249 shutdown 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 msgr2=0x55eac76d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76d0f00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c140ad230 tx=0x7f3c140e4ff0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 shutdown_connections 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76d0f00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72d18e0 0x55eac76d7600 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1244710533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 >> 172.21.15.27:0/1244710533 conn(0x55eac7307350 msgr2=0x55eac76bc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c03400640 0 client.15249 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 shutdown_connections 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.334+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1244710533 wait complete. 2024-09-21T01:21:36.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.335+0000 7f3c2ae01740 20 client.15249 trim_cache size 0 max 16384 2024-09-21T01:21:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.353+0000 7f3c2ae01740 1 Processor -- start 2024-09-21T01:21:36.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 -- start start 2024-09-21T01:21:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 0x55eac76d7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72f2280 0x55eac76c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eac76d5940 con 0x55eac72ea0a0 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eac76d6010 con 0x55eac7312f60 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eac76d66e0 con 0x55eac72f2280 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c21800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 0x55eac76d7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c20e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72f2280 0x55eac76c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c21800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 0x55eac76d7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36948/0 (socket says 172.21.15.27:36948) 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c20e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72f2280 0x55eac76c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51668/0 (socket says 172.21.15.27:51668) 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c21800640 1 -- 172.21.15.27:0/3851264116 learned_addr learned my addr 172.21.15.27:0/3851264116 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.354+0000 7f3c22200640 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c22200640 1 -- 172.21.15.27:0/3851264116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72f2280 msgr2=0x55eac76c0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c22200640 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72f2280 0x55eac76c0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c22200640 1 -- 172.21.15.27:0/3851264116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 msgr2=0x55eac76d7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c22200640 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 0x55eac76d7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c22200640 1 -- 172.21.15.27:0/3851264116 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eac76c7600 con 0x55eac7312f60 2024-09-21T01:21:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c21800640 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 0x55eac76d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:36.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c22200640 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76c05b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c14005c40 tx=0x7f3c14003fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84eda22006a99125 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c0e000640 1 -- 172.21.15.27:0/3851264116 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14020070 con 0x55eac7312f60 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c0e000640 1 -- 172.21.15.27:0/3851264116 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c140e6070 con 0x55eac7312f60 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.355+0000 7f3c0e000640 1 -- 172.21.15.27:0/3851264116 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c140d0b70 con 0x55eac7312f60 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3851264116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 msgr2=0x55eac76c05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76c05b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c14005c40 tx=0x7f3c14003fd0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3851264116 shutdown_connections 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac7312f60 0x55eac76c05b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72f2280 0x55eac76c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/3851264116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac72ea0a0 0x55eac76d7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3851264116 >> 172.21.15.27:0/3851264116 conn(0x55eac7307350 msgr2=0x55eac76b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3851264116 shutdown_connections 2024-09-21T01:21:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.356+0000 7f3c2ae01740 1 -- 172.21.15.27:0/3851264116 wait complete. 2024-09-21T01:21:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.357+0000 7f3c2ae01740 1 -- start start 2024-09-21T01:21:36.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.357+0000 7f3c2ae01740 1 Processor -- start 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76de450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55eac76d7480 con 0x55eac7312f60 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55eac76d6db0 con 0x55eac72f2280 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eac76d7b50 con 0x55eac72ea0a0 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c2ae01740 10 client.0 _ll_register_callbacks cb 0x7f3c140ad070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c22200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76de450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c22200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76de450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36958/0 (socket says 172.21.15.27:36958) 2024-09-21T01:21:36.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c22200640 1 -- 172.21.15.27:0/1062727547 learned_addr learned my addr 172.21.15.27:0/1062727547 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c20e00640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.358+0000 7f3c21800640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c22200640 1 -- 172.21.15.27:0/1062727547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 msgr2=0x55eac76d9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c22200640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76d9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c22200640 1 -- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 msgr2=0x55eac76d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c22200640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76d95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c22200640 1 -- 172.21.15.27:0/1062727547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eac76deac0 con 0x55eac7312f60 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c22200640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76de450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c14004060 tx=0x7f3c140ad140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a90087f621ce35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c03400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c03400640 1 -- 172.21.15.27:0/1062727547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14020070 con 0x55eac7312f60 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c03400640 1 -- 172.21.15.27:0/1062727547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3c14058ae0 con 0x55eac7312f60 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.359+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c140e6070 con 0x55eac7312f60 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c03400640 1 -- 172.21.15.27:0/1062727547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c140b1cd0 con 0x55eac7312f60 2024-09-21T01:21:36.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 10 client.15255 fetch_fsmap learned FSMap version 28 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eac76dfe80 con 0x55eac7312f60 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c03400640 1 -- 172.21.15.27:0/1062727547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c140e6070 con 0x55eac7312f60 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c03400640 1 -- 172.21.15.27:0/1062727547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3c140abf70 con 0x55eac7312f60 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 10 client.15255 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 20 client.15255 populate_metadata read hostname 'smithi027' 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72a6c90 0x55eac7745550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 4 client.15255 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:21:36.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.360+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "status"}) -- 0x55eac762e750 con 0x55eac72a6c90 2024-09-21T01:21:36.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.361+0000 7f3c21800640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72a6c90 0x55eac7745550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:36.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.361+0000 7f3c21800640 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72a6c90 0x55eac7745550 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7953152a961289e server_cookie=9afe6250f9da48f1 in_seq=0 out_seq=0 2024-09-21T01:21:36.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.361+0000 7f3c03400640 10 client.15255 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c03400640 1 -- 172.21.15.27:0/1062727547 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eac762e750 con 0x55eac72a6c90 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c03400640 10 client.15255 handle_command_reply: tid=0 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 client.15255 shutdown 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 msgr2=0x55eac76de450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76de450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3c14004060 tx=0x7f3c140ad140 comp rx=0 tx=0).stop 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 shutdown_connections 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55eac7312f60 0x55eac76de450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55eac72a6c90 0x55eac7745550 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55eac72f2280 0x55eac76d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 --2- 172.21.15.27:0/1062727547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eac72ea0a0 0x55eac76d9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c03400640 0 client.15255 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 >> 172.21.15.27:0/1062727547 conn(0x55eac7307350 msgr2=0x55eac76c3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:36.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 shutdown_connections 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 1 -- 172.21.15.27:0/1062727547 wait complete. 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.362+0000 7f3c2ae01740 20 client.15255 trim_cache size 0 max 16384 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24776, 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:21:36.366 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 147.63191991599999, 2024-09-21T01:21:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:21:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:21:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:21:36.367 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 162.13985999900001 2024-09-21T01:21:36.367 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:21:36.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c040b65f0 msgr2=0x7f3c040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c040b65f0 0x7f3c040b8b00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c080058c0 tx=0x7f3c08005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 msgr2=0x7f3c1c1a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c1a1ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009fd0 tx=0x7f3c1400b260 comp rx=0 tx=0).stop 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 shutdown_connections 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3c040b65f0 0x7f3c040b8b00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3c1c110140 0x7f3c1c1a1ee0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c1c076080 0x7f3c1c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 --2- 172.21.15.27:0/2597871684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c1c075730 0x7f3c1c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 >> 172.21.15.27:0/2597871684 conn(0x7f3c1c07b850 msgr2=0x7f3c1c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 shutdown_connections 2024-09-21T01:21:36.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:36.363+0000 7f3c24200640 1 -- 172.21.15.27:0/2597871684 wait complete. 2024-09-21T01:21:36.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:36 smithi098 ceph-mon[34372]: pgmap v375: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:36 smithi089 ceph-mon[34486]: pgmap v375: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:21:37.184 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T01:21:37.184 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T01:21:37.199 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:21:37.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T01:21:37.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:37 smithi027 ceph-mon[30375]: pgmap v376: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:37.585 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:37.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:37 smithi098 ceph-mon[34372]: pgmap v376: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:37.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:37 smithi089 ceph-mon[34486]: pgmap v376: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 -- 172.21.15.27:0/389073339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe074105db0 msgr2=0x7fe0741061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 --2- 172.21.15.27:0/389073339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe074105db0 0x7fe0741061c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe068009900 tx=0x7fe06802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 -- 172.21.15.27:0/389073339 shutdown_connections 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 --2- 172.21.15.27:0/389073339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe07410c040 0x7fe074112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 --2- 172.21.15.27:0/389073339 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe07410b6f0 0x7fe07410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 --2- 172.21.15.27:0/389073339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe074105db0 0x7fe0741061c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 -- 172.21.15.27:0/389073339 >> 172.21.15.27:0/389073339 conn(0x7fe0741017e0 msgr2=0x7fe074103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 -- 172.21.15.27:0/389073339 shutdown_connections 2024-09-21T01:21:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.226+0000 7fe07c400640 1 -- 172.21.15.27:0/389073339 wait complete. 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.227+0000 7fe07c400640 1 Processor -- start 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 -- start start 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 0x7fe07419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07410b6f0 0x7fe07419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe07410c040 0x7fe0741a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe074115d00 con 0x7fe07410b6f0 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe074115b80 con 0x7fe074105db0 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe07c400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe074115e80 con 0x7fe07410c040 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe079a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 0x7fe07419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe079000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07410b6f0 0x7fe07419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe079a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 0x7fe07419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49094/0 (socket says 172.21.15.27:49094) 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.228+0000 7fe079a00640 1 -- 172.21.15.27:0/4068549034 learned_addr learned my addr 172.21.15.27:0/4068549034 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe07a400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe07410c040 0x7fe0741a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079000640 1 -- 172.21.15.27:0/4068549034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe07410c040 msgr2=0x7fe0741a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079000640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe07410c040 0x7fe0741a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079000640 1 -- 172.21.15.27:0/4068549034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 msgr2=0x7fe07419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079000640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 0x7fe07419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079000640 1 -- 172.21.15.27:0/4068549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0741a2660 con 0x7fe07410b6f0 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079a00640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 0x7fe07419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.229+0000 7fe079000640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07410b6f0 0x7fe07419cc90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe07406ca20 tx=0x7fe06000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.230+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06000cd00 con 0x7fe07410b6f0 2024-09-21T01:21:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.230+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0600035f0 con 0x7fe07410b6f0 2024-09-21T01:21:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.230+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0741a2950 con 0x7fe07410b6f0 2024-09-21T01:21:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.230+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060010690 con 0x7fe07410b6f0 2024-09-21T01:21:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.230+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0741a90b0 con 0x7fe07410b6f0 2024-09-21T01:21:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.231+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0741061c0 con 0x7fe07410b6f0 2024-09-21T01:21:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.234+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7fe0600037a0 con 0x7fe07410b6f0 2024-09-21T01:21:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.235+0000 7fe072000640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe05c0b65f0 0x7fe05c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.235+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe0600e7dd0 con 0x7fe07410b6f0 2024-09-21T01:21:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.235+0000 7fe079a00640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe05c0b65f0 0x7fe05c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.236+0000 7fe079a00640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe05c0b65f0 0x7fe05c0b8b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe068031440 tx=0x7fe0680058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.237+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe060056710 con 0x7fe07410b6f0 2024-09-21T01:21:38.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.453+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe074063800 con 0x7fe05c0b65f0 2024-09-21T01:21:38.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.455+0000 7fe072000640 1 -- 172.21.15.27:0/4068549034 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe074063800 con 0x7fe05c0b65f0 2024-09-21T01:21:38.455 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2024-09-21T01:21:38.456 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2024-09-21T01:21:38.456 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe05c0b65f0 msgr2=0x7fe05c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe05c0b65f0 0x7fe05c0b8b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe068031440 tx=0x7fe0680058d0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07410b6f0 msgr2=0x7fe07419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07410b6f0 0x7fe07419cc90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe07406ca20 tx=0x7fe06000ee50 comp rx=0 tx=0).stop 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 shutdown_connections 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe05c0b65f0 0x7fe05c0b8b00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe07410c040 0x7fe0741a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe07410b6f0 0x7fe07419cc90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 --2- 172.21.15.27:0/4068549034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe074105db0 0x7fe07419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:38.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 >> 172.21.15.27:0/4068549034 conn(0x7fe0741017e0 msgr2=0x7fe07410ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:38.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 shutdown_connections 2024-09-21T01:21:38.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:38.457+0000 7fe07c400640 1 -- 172.21.15.27:0/4068549034 wait complete. 2024-09-21T01:21:39.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T01:21:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:39 smithi027 ceph-mon[30375]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:39.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:39 smithi027 ceph-mon[30375]: pgmap v377: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:39.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:39 smithi098 ceph-mon[34372]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:39.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:39 smithi098 ceph-mon[34372]: pgmap v377: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:39.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:39 smithi089 ceph-mon[34486]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:40.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:39 smithi089 ceph-mon[34486]: pgmap v377: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 -- 172.21.15.27:0/3482489902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810b6c0 msgr2=0x7f6af810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 --2- 172.21.15.27:0/3482489902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810b6c0 0x7f6af810bad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009900 tx=0x7f6ae402a410 comp rx=0 tx=0).stop 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 -- 172.21.15.27:0/3482489902 shutdown_connections 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 --2- 172.21.15.27:0/3482489902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af810c010 0x7f6af8112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 --2- 172.21.15.27:0/3482489902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810b6c0 0x7f6af810bad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 --2- 172.21.15.27:0/3482489902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af8105d80 0x7f6af8106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 -- 172.21.15.27:0/3482489902 >> 172.21.15.27:0/3482489902 conn(0x7f6af8101810 msgr2=0x7f6af8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:40.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 -- 172.21.15.27:0/3482489902 shutdown_connections 2024-09-21T01:21:40.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.428+0000 7f6affe00640 1 -- 172.21.15.27:0/3482489902 wait complete. 2024-09-21T01:21:40.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.429+0000 7f6affe00640 1 Processor -- start 2024-09-21T01:21:40.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.429+0000 7f6affe00640 1 -- start start 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6affe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af8105d80 0x7f6af819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6affe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 0x7f6af819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6affe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810c010 0x7f6af81a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6affe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8115cd0 con 0x7f6af810c010 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6affe00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8115b50 con 0x7f6af810b6c0 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6affe00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8115e50 con 0x7f6af8105d80 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6afca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 0x7f6af819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6afca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 0x7f6af819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49116/0 (socket says 172.21.15.27:49116) 2024-09-21T01:21:40.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6afca00640 1 -- 172.21.15.27:0/3047791265 learned_addr learned my addr 172.21.15.27:0/3047791265 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6afd400640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af8105d80 0x7f6af819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.430+0000 7f6afde00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810c010 0x7f6af81a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6afca00640 1 -- 172.21.15.27:0/3047791265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af8105d80 msgr2=0x7f6af819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6afca00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af8105d80 0x7f6af819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6afca00640 1 -- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810c010 msgr2=0x7f6af81a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6afca00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810c010 0x7f6af81a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6afca00640 1 -- 172.21.15.27:0/3047791265 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af81a2610 con 0x7f6af810b6c0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6afca00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 0x7f6af819cc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6ae402a930 tx=0x7f6ae4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae403f030 con 0x7f6af810b6c0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae4004490 con 0x7f6af810b6c0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af81a28a0 con 0x7f6af810b6c0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.431+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4035070 con 0x7f6af810b6c0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.432+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a9060 con 0x7f6af810b6c0 2024-09-21T01:21:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.433+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af8107000 con 0x7f6af810b6c0 2024-09-21T01:21:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.435+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f6ae4042030 con 0x7f6af810b6c0 2024-09-21T01:21:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.437+0000 7f6af6000640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ae00b65f0 0x7f6ae00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.437+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6ae41080c0 con 0x7f6af810b6c0 2024-09-21T01:21:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.437+0000 7f6afd400640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ae00b65f0 0x7f6ae00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:40.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.438+0000 7f6afd400640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ae00b65f0 0x7f6ae00b8b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009bd0 tx=0x7f6aec005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:40.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.439+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ae40d26e0 con 0x7f6af810b6c0 2024-09-21T01:21:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.650+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6af8063800 con 0x7f6ae00b65f0 2024-09-21T01:21:40.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.665+0000 7f6af6000640 1 -- 172.21.15.27:0/3047791265 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6af8063800 con 0x7f6ae00b65f0 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.a smithi027 running (2m) 26s ago 2m 80.6M - 19.3.0-5120-g18432218 ddce307e1f32 a619101c83b8 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.b smithi089 running (2m) 33s ago 2m 94.0M - 19.3.0-5120-g18432218 ddce307e1f32 2867b76e2e4f 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.c smithi098 running (3m) 33s ago 3m 86.1M - 19.3.0-5120-g18432218 ddce307e1f32 124e199b8808 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.d smithi027 running (3m) 26s ago 3m 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 2ac635b2c344 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.e smithi089 running (3m) 33s ago 3m 89.3M - 19.3.0-5120-g18432218 ddce307e1f32 eac84a7a3c52 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.f smithi098 running (3m) 33s ago 3m 88.0M - 19.3.0-5120-g18432218 ddce307e1f32 cd07f5337298 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.g smithi027 running (2m) 26s ago 2m 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 0d3e5d93f44c 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.h smithi089 running (3m) 33s ago 2m 86.6M - 19.3.0-5120-g18432218 ddce307e1f32 87025dbafe51 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.i smithi098 running (2m) 33s ago 2m 86.8M - 19.3.0-5120-g18432218 ddce307e1f32 5edfd74fe830 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.j smithi027 running (3m) 26s ago 3m 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 612eec67dae2 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.k smithi089 running (3m) 33s ago 3m 85.9M - 19.3.0-5120-g18432218 ddce307e1f32 8e78bce89c24 2024-09-21T01:21:40.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.l smithi098 running (3m) 33s ago 3m 82.4M - 19.3.0-5120-g18432218 ddce307e1f32 72874aa4d923 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.x smithi027 *:9283,8765 running (11m) 26s ago 11m 587M - 19.3.0-5120-g18432218 ddce307e1f32 a032899fd50c 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.y smithi089 *:8443,8765 running (9m) 33s ago 9m 524M - 19.3.0-5120-g18432218 ddce307e1f32 789eb5d1a428 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.z smithi098 *:8443,8765 running (9m) 33s ago 9m 524M - 19.3.0-5120-g18432218 ddce307e1f32 3a4296d6352b 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:mon.a smithi027 running (11m) 26s ago 12m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 14af1170be19 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:mon.b smithi089 running (10m) 33s ago 10m 112M 2048M 19.3.0-5120-g18432218 ddce307e1f32 2258882a6026 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:mon.c smithi098 running (10m) 33s ago 10m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 ff8ee050bf7a 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 26s ago 8m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 4438aacc1a14 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi089 running (8m) 33s ago 8m 114M 4271M 19.3.0-5120-g18432218 ddce307e1f32 0612ce39ba73 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi098 running (7m) 33s ago 7m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 038ce13d32fe 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi027 running (7m) 26s ago 7m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 846118b2aaae 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi089 running (7m) 33s ago 6m 122M 4271M 19.3.0-5120-g18432218 ddce307e1f32 4829958b29f3 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi098 running (6m) 33s ago 6m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 4487c12bfc08 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 smithi027 running (6m) 26s ago 6m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 b336281eaed9 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 smithi089 running (5m) 33s ago 5m 121M 4271M 19.3.0-5120-g18432218 ddce307e1f32 ab421179735e 2024-09-21T01:21:40.667 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 smithi098 running (5m) 33s ago 5m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 bbd3032f6b87 2024-09-21T01:21:40.668 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 smithi027 running (4m) 26s ago 4m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c58f43a4a51f 2024-09-21T01:21:40.668 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 smithi089 running (4m) 33s ago 4m 114M 4271M 19.3.0-5120-g18432218 ddce307e1f32 3d88876e89a6 2024-09-21T01:21:40.668 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 smithi098 running (3m) 33s ago 3m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 74d477139946 2024-09-21T01:21:40.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ae00b65f0 msgr2=0x7f6ae00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ae00b65f0 0x7f6ae00b8b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009bd0 tx=0x7f6aec005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 msgr2=0x7f6af819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 0x7f6af819cc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6ae402a930 tx=0x7f6ae4031a20 comp rx=0 tx=0).stop 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 shutdown_connections 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6ae00b65f0 0x7f6ae00b8b00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6af810c010 0x7f6af81a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6af810b6c0 0x7f6af819cc40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 --2- 172.21.15.27:0/3047791265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6af8105d80 0x7f6af819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.667+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 >> 172.21.15.27:0/3047791265 conn(0x7f6af8101810 msgr2=0x7f6af8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.668+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 shutdown_connections 2024-09-21T01:21:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:40.668+0000 7f6affe00640 1 -- 172.21.15.27:0/3047791265 wait complete. 2024-09-21T01:21:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:41 smithi027 ceph-mon[30375]: pgmap v378: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:41.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:41 smithi027 ceph-mon[30375]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:41.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T01:21:41.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:41 smithi098 ceph-mon[34372]: pgmap v378: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:41.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:41 smithi098 ceph-mon[34372]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:42.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:41 smithi089 ceph-mon[34486]: pgmap v378: 129 pgs: 129 active+clean; 648 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:21:42.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:41 smithi089 ceph-mon[34486]: from='client.25310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 -- 172.21.15.27:0/2328909766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 msgr2=0x7f31ac112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 --2- 172.21.15.27:0/2328909766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 0x7f31ac112840 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009ca0 tx=0x7f31a402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 -- 172.21.15.27:0/2328909766 shutdown_connections 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 --2- 172.21.15.27:0/2328909766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 0x7f31ac112840 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 --2- 172.21.15.27:0/2328909766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 0x7f31ac10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 --2- 172.21.15.27:0/2328909766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 -- 172.21.15.27:0/2328909766 >> 172.21.15.27:0/2328909766 conn(0x7f31ac1017d0 msgr2=0x7f31ac103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 -- 172.21.15.27:0/2328909766 shutdown_connections 2024-09-21T01:21:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.913+0000 7f31b3c00640 1 -- 172.21.15.27:0/2328909766 wait complete. 2024-09-21T01:21:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.914+0000 7f31b3c00640 1 Processor -- start 2024-09-21T01:21:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.914+0000 7f31b3c00640 1 -- start start 2024-09-21T01:21:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b3c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b3c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 0x7f31ac19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b3c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 0x7f31ac1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b3c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac115cd0 con 0x7f31ac10c010 2024-09-21T01:21:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b3c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac115b50 con 0x7f31ac10b6c0 2024-09-21T01:21:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b3c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac115e50 con 0x7f31ac105d80 2024-09-21T01:21:42.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b1200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51710/0 (socket says 172.21.15.27:51710) 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b1200640 1 -- 172.21.15.27:0/1792662384 learned_addr learned my addr 172.21.15.27:0/1792662384 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31b1c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 0x7f31ac1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.915+0000 7f31abe00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 0x7f31ac19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31abe00640 1 -- 172.21.15.27:0/1792662384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 msgr2=0x7f31ac19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31abe00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31abe00640 1 -- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 msgr2=0x7f31ac1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31abe00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 0x7f31ac1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31abe00640 1 -- 172.21.15.27:0/1792662384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac1a25b0 con 0x7f31ac10b6c0 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31b1200640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31abe00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 0x7f31ac19cbe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31ac06cac0 tx=0x7f319800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319800cc40 con 0x7f31ac10b6c0 2024-09-21T01:21:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f319800cde0 con 0x7f31ac10b6c0 2024-09-21T01:21:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac1a28a0 con 0x7f31ac10b6c0 2024-09-21T01:21:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.916+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198018780 con 0x7f31ac10b6c0 2024-09-21T01:21:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.917+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac1a9000 con 0x7f31ac10b6c0 2024-09-21T01:21:42.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.918+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ac106190 con 0x7f31ac10b6c0 2024-09-21T01:21:42.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.921+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 154580+0+0 (secure 0 0 0) 0x7f3198010a90 con 0x7f31ac10b6c0 2024-09-21T01:21:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.923+0000 7f31a9600640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f31940b6670 0x7f31940b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.923+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f31980e8060 con 0x7f31ac10b6c0 2024-09-21T01:21:42.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.923+0000 7f31b1200640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f31940b6670 0x7f31940b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:42.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.924+0000 7f31b1200640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f31940b6670 0x7f31940b8b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f31a00095f0 tx=0x7f31a0005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:42.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:42.924+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31980b26e0 con 0x7f31ac10b6c0 2024-09-21T01:21:43.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.131+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f31ac063800 con 0x7f31940b6670 2024-09-21T01:21:43.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.137+0000 7f31a9600640 1 -- 172.21.15.27:0/1792662384 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f31ac063800 con 0x7f31940b6670 2024-09-21T01:21:43.137 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T01:21:43.138 INFO:teuthology.orchestra.run.smithi027.stdout:mds.all 12/12 36s ago 3m smithi027=a;smithi027=d;smithi027=g;smithi027=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi098=c;smithi098=f;smithi098=i;smithi098=l;count:12 2024-09-21T01:21:43.138 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 3/3 36s ago 10m smithi027=x;smithi089=y;smithi098=z;count:3 2024-09-21T01:21:43.138 INFO:teuthology.orchestra.run.smithi027.stdout:mon 3/3 36s ago 10m smithi027:172.21.15.27=a;smithi089:172.21.15.89=b;smithi098:172.21.15.98=c;count:3 2024-09-21T01:21:43.138 INFO:teuthology.orchestra.run.smithi027.stdout:osd 12 36s ago - 2024-09-21T01:21:43.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f31940b6670 msgr2=0x7f31940b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:43.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f31940b6670 0x7f31940b8b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f31a00095f0 tx=0x7f31a0005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 msgr2=0x7f31ac19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 0x7f31ac19cbe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31ac06cac0 tx=0x7f319800ee50 comp rx=0 tx=0).stop 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 shutdown_connections 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f31940b6670 0x7f31940b8b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31ac10c010 0x7f31ac1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f31ac10b6c0 0x7f31ac19cbe0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 --2- 172.21.15.27:0/1792662384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31ac105d80 0x7f31ac19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.139+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 >> 172.21.15.27:0/1792662384 conn(0x7f31ac1017d0 msgr2=0x7f31ac103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.140+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 shutdown_connections 2024-09-21T01:21:43.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:43.140+0000 7f31b3c00640 1 -- 172.21.15.27:0/1792662384 wait complete. 2024-09-21T01:21:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:43 smithi098 ceph-mon[34372]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:21:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:43 smithi098 ceph-mon[34372]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:43.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:43 smithi027 ceph-mon[30375]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:21:43.909 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:43 smithi027 ceph-mon[30375]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:43.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T01:21:43.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:43 smithi089 ceph-mon[34486]: pgmap v379: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:21:44.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:43 smithi089 ceph-mon[34486]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:44.332 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:44.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:44 smithi027 ceph-mon[30375]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-21T01:21:44.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:44 smithi098 ceph-mon[34372]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 -- 172.21.15.27:0/170166892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1092f0 msgr2=0x7f731c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 --2- 172.21.15.27:0/170166892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1092f0 0x7f731c109700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7308009900 tx=0x7f730802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 -- 172.21.15.27:0/170166892 shutdown_connections 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 --2- 172.21.15.27:0/170166892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c109c40 0x7f731c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 --2- 172.21.15.27:0/170166892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1092f0 0x7f731c109700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 --2- 172.21.15.27:0/170166892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f731c1039b0 0x7f731c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 -- 172.21.15.27:0/170166892 >> 172.21.15.27:0/170166892 conn(0x7f731c0ff400 msgr2=0x7f731c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 -- 172.21.15.27:0/170166892 shutdown_connections 2024-09-21T01:21:44.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.990+0000 7f731b200640 1 -- 172.21.15.27:0/170166892 wait complete. 2024-09-21T01:21:44.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.991+0000 7f731b200640 1 Processor -- start 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.991+0000 7f731b200640 1 -- start start 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1039b0 0x7f731c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 0x7f731c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f731c109c40 0x7f731c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f731c113900 con 0x7f731c1039b0 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f731c113780 con 0x7f731c109c40 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f731c113a80 con 0x7f731c1092f0 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 0x7f731c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1039b0 0x7f731c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 0x7f731c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:51730/0 (socket says 172.21.15.27:51730) 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319400640 1 -- 172.21.15.27:0/1552535096 learned_addr learned my addr 172.21.15.27:0/1552535096 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:44.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f731a800640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f731c109c40 0x7f731c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319400640 1 -- 172.21.15.27:0/1552535096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f731c109c40 msgr2=0x7f731c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319400640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f731c109c40 0x7f731c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:44.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.992+0000 7f7319400640 1 -- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1039b0 msgr2=0x7f731c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f7319400640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1039b0 0x7f731c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f7319400640 1 -- 172.21.15.27:0/1552535096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c1a85b0 con 0x7f731c1092f0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f7319400640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 0x7f731c075c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f730802a910 tx=0x7f7308031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730803f030 con 0x7f731c1092f0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7308004490 con 0x7f731c1092f0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308035070 con 0x7f731c1092f0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c1a8840 con 0x7f731c1092f0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.993+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c1a8d20 con 0x7f731c1092f0 2024-09-21T01:21:44.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.994+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c103dc0 con 0x7f731c1092f0 2024-09-21T01:21:44.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.997+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f7308042030 con 0x7f731c1092f0 2024-09-21T01:21:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.999+0000 7f7312a00640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73000b6560 0x7f73000b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.999+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7308107f20 con 0x7f731c1092f0 2024-09-21T01:21:44.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:44.999+0000 7f7319e00640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73000b6560 0x7f73000b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:45.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:44 smithi089 ceph-mon[34486]: mgrmap e23: x(active, since 10m), standbys: z, y 2024-09-21T01:21:45.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.000+0000 7f7319e00640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73000b6560 0x7f73000b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f731c077160 tx=0x7f730c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:45.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.000+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f73080d26a0 con 0x7f731c1092f0 2024-09-21T01:21:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.208+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f731c063800 con 0x7f73000b6560 2024-09-21T01:21:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.209+0000 7f7312a00640 1 -- 172.21.15.27:0/1552535096 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f731c063800 con 0x7f73000b6560 2024-09-21T01:21:45.210 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2024-09-21T01:21:45.210 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2024-09-21T01:21:45.210 INFO:teuthology.orchestra.run.smithi027.stdout:smithi089 172.21.15.89 2024-09-21T01:21:45.210 INFO:teuthology.orchestra.run.smithi027.stdout:smithi098 172.21.15.98 2024-09-21T01:21:45.210 INFO:teuthology.orchestra.run.smithi027.stdout:3 hosts in cluster 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73000b6560 msgr2=0x7f73000b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73000b6560 0x7f73000b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f731c077160 tx=0x7f730c005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 msgr2=0x7f731c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 0x7f731c075c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f730802a910 tx=0x7f7308031a20 comp rx=0 tx=0).stop 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 shutdown_connections 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f73000b6560 0x7f73000b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f731c109c40 0x7f731c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731c1092f0 0x7f731c075c50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 --2- 172.21.15.27:0/1552535096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f731c1039b0 0x7f731c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 >> 172.21.15.27:0/1552535096 conn(0x7f731c0ff400 msgr2=0x7f731c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 shutdown_connections 2024-09-21T01:21:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:45.212+0000 7f731b200640 1 -- 172.21.15.27:0/1552535096 wait complete. 2024-09-21T01:21:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:45 smithi098 ceph-mon[34372]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T01:21:45.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:45 smithi027 ceph-mon[30375]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T01:21:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:45 smithi089 ceph-mon[34486]: pgmap v380: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T01:21:46.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T01:21:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:46.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:46 smithi098 ceph-mon[34372]: from='client.25309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:46.918 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:46 smithi027 ceph-mon[30375]: from='client.25309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:46.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:46 smithi089 ceph-mon[34486]: from='client.25309 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 -- 172.21.15.27:0/556343263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c105d80 msgr2=0x7f620c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 --2- 172.21.15.27:0/556343263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c105d80 0x7f620c106190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6200009900 tx=0x7f620002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 -- 172.21.15.27:0/556343263 shutdown_connections 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 --2- 172.21.15.27:0/556343263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10c010 0x7f620c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 --2- 172.21.15.27:0/556343263 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c10b6c0 0x7f620c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 --2- 172.21.15.27:0/556343263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c105d80 0x7f620c106190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 -- 172.21.15.27:0/556343263 >> 172.21.15.27:0/556343263 conn(0x7f620c101810 msgr2=0x7f620c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 -- 172.21.15.27:0/556343263 shutdown_connections 2024-09-21T01:21:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.268+0000 7f6214200640 1 -- 172.21.15.27:0/556343263 wait complete. 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.269+0000 7f6214200640 1 Processor -- start 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.269+0000 7f6214200640 1 -- start start 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6214200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c105d80 0x7f620c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6214200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 0x7f620c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6214200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c10c010 0x7f620c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6214200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f620c115cd0 con 0x7f620c10c010 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6214200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f620c115b50 con 0x7f620c105d80 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6214200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f620c115e50 con 0x7f620c10b6c0 2024-09-21T01:21:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6211800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c105d80 0x7f620c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6210e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 0x7f620c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6212200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c10c010 0x7f620c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6210e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 0x7f620c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38954/0 (socket says 172.21.15.27:38954) 2024-09-21T01:21:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.270+0000 7f6210e00640 1 -- 172.21.15.27:0/2102258644 learned_addr learned my addr 172.21.15.27:0/2102258644 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6210e00640 1 -- 172.21.15.27:0/2102258644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c105d80 msgr2=0x7f620c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6210e00640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c105d80 0x7f620c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6210e00640 1 -- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c10c010 msgr2=0x7f620c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6210e00640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c10c010 0x7f620c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6210e00640 1 -- 172.21.15.27:0/2102258644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c1a2630 con 0x7f620c10b6c0 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6212200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c10c010 0x7f620c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6210e00640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 0x7f620c19cc60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e9f0 tx=0x7f61f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f800ccc0 con 0x7f620c10b6c0 2024-09-21T01:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c1a2920 con 0x7f620c10b6c0 2024-09-21T01:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61f80108e0 con 0x7f620c10b6c0 2024-09-21T01:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8018860 con 0x7f620c10b6c0 2024-09-21T01:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.271+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c1a9080 con 0x7f620c10b6c0 2024-09-21T01:21:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.272+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c106190 con 0x7f620c10b6c0 2024-09-21T01:21:47.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.275+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f61f8010430 con 0x7f620c10b6c0 2024-09-21T01:21:47.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.278+0000 7f620a000640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f61f40b6560 0x7f61f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:47.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.278+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f61f80e7f90 con 0x7f620c10b6c0 2024-09-21T01:21:47.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.278+0000 7f6211800640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f61f40b6560 0x7f61f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:47.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.278+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61f80b2710 con 0x7f620c10b6c0 2024-09-21T01:21:47.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.279+0000 7f6211800640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f61f40b6560 0x7f61f40b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6200031440 tx=0x7f6200009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:47.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.490+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f620c10de10 con 0x7f61f40b6560 2024-09-21T01:21:47.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.492+0000 7f620a000640 1 -- 172.21.15.27:0/2102258644 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f620c10de10 con 0x7f61f40b6560 2024-09-21T01:21:47.493 INFO:teuthology.orchestra.run.smithi027.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T01:21:47.493 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-21T01:21:47.493 INFO:teuthology.orchestra.run.smithi027.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-21T01:21:47.493 INFO:teuthology.orchestra.run.smithi027.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T01:21:47.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.494+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f61f40b6560 msgr2=0x7f61f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:47.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.494+0000 7f6214200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f61f40b6560 0x7f61f40b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6200031440 tx=0x7f6200009560 comp rx=0 tx=0).stop 2024-09-21T01:21:47.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.494+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 msgr2=0x7f620c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.494+0000 7f6214200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 0x7f620c19cc60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e9f0 tx=0x7f61f800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.494+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 shutdown_connections 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.494+0000 7f6214200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f61f40b6560 0x7f61f40b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.495+0000 7f6214200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f620c10c010 0x7f620c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.495+0000 7f6214200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f620c10b6c0 0x7f620c19cc60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.495+0000 7f6214200640 1 --2- 172.21.15.27:0/2102258644 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f620c105d80 0x7f620c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.495+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 >> 172.21.15.27:0/2102258644 conn(0x7f620c101810 msgr2=0x7f620c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.495+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 shutdown_connections 2024-09-21T01:21:47.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:47.495+0000 7f6214200640 1 -- 172.21.15.27:0/2102258644 wait complete. 2024-09-21T01:21:47.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:47 smithi098 ceph-mon[34372]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T01:21:47.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:47 smithi027 ceph-mon[30375]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T01:21:48.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:47 smithi089 ceph-mon[34486]: pgmap v381: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-21T01:21:48.369 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T01:21:48.378 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:21:48.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T01:21:48.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:48 smithi027 ceph-mon[30375]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:48.766 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:48.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:48 smithi098 ceph-mon[34372]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:49.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:48 smithi089 ceph-mon[34486]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:21:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 -- 172.21.15.27:0/4074330217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba4010b6c0 msgr2=0x7fba4010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 --2- 172.21.15.27:0/4074330217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba4010b6c0 0x7fba4010bad0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba28009900 tx=0x7fba2802a410 comp rx=0 tx=0).stop 2024-09-21T01:21:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 -- 172.21.15.27:0/4074330217 shutdown_connections 2024-09-21T01:21:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 --2- 172.21.15.27:0/4074330217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba4010c010 0x7fba40112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 --2- 172.21.15.27:0/4074330217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba4010b6c0 0x7fba4010bad0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 --2- 172.21.15.27:0/4074330217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba40105d80 0x7fba40106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.468+0000 7fba45000640 1 -- 172.21.15.27:0/4074330217 >> 172.21.15.27:0/4074330217 conn(0x7fba40101810 msgr2=0x7fba40103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.469+0000 7fba45000640 1 -- 172.21.15.27:0/4074330217 shutdown_connections 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.469+0000 7fba45000640 1 -- 172.21.15.27:0/4074330217 wait complete. 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.470+0000 7fba45000640 1 Processor -- start 2024-09-21T01:21:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.470+0000 7fba45000640 1 -- start start 2024-09-21T01:21:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.470+0000 7fba45000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba40105d80 0x7fba4019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba45000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 0x7fba4019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba45000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba4010c010 0x7fba401a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba45000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fba40115d20 con 0x7fba40105d80 2024-09-21T01:21:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba45000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fba40115ba0 con 0x7fba4010c010 2024-09-21T01:21:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba45000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fba40115ea0 con 0x7fba4010b6c0 2024-09-21T01:21:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba3de00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba40105d80 0x7fba4019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 0x7fba4019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 0x7fba4019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38968/0 (socket says 172.21.15.27:38968) 2024-09-21T01:21:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 -- 172.21.15.27:0/120274959 learned_addr learned my addr 172.21.15.27:0/120274959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba3e800640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba4010c010 0x7fba401a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 -- 172.21.15.27:0/120274959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba4010c010 msgr2=0x7fba401a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba4010c010 0x7fba401a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 -- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba40105d80 msgr2=0x7fba4019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba40105d80 0x7fba4019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba35400640 1 -- 172.21.15.27:0/120274959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba401a2660 con 0x7fba4010b6c0 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.471+0000 7fba3de00640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba40105d80 0x7fba4019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.472+0000 7fba35400640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 0x7fba4019cc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba28005ab0 tx=0x7fba28003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.472+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2803f070 con 0x7fba4010b6c0 2024-09-21T01:21:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.472+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba28031b50 con 0x7fba4010b6c0 2024-09-21T01:21:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.472+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba401a28f0 con 0x7fba4010b6c0 2024-09-21T01:21:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.472+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28035030 con 0x7fba4010b6c0 2024-09-21T01:21:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.472+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba401a90b0 con 0x7fba4010b6c0 2024-09-21T01:21:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.473+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba40107000 con 0x7fba4010b6c0 2024-09-21T01:21:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.476+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fba28042070 con 0x7fba4010b6c0 2024-09-21T01:21:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.478+0000 7fba37400640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba240b6560 0x7fba240b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.478+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fba28108640 con 0x7fba4010b6c0 2024-09-21T01:21:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.478+0000 7fba3de00640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba240b6560 0x7fba240b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.479+0000 7fba3de00640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba240b6560 0x7fba240b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba30009fc0 tx=0x7fba30005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.479+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba280d2dc0 con 0x7fba4010b6c0 2024-09-21T01:21:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.714+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fba40106190 con 0x7fba4010b6c0 2024-09-21T01:21:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.715+0000 7fba37400640 1 -- 172.21.15.27:0/120274959 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3120 (secure 0 0 0) 0x7fba280d7d00 con 0x7fba4010b6c0 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:btime 2024-09-21T01:20:08:397428+0000 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2024-09-21T01:21:49.716 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-21T01:19:07.935504+0000 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-21T01:20:07.570104+0000 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 300 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:max_xattr_size 65536 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2024-09-21T01:21:49.717 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 5 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1,2,3,4 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24776,1=24752,2=24734,3=24730,4=24718} 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3,4] 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:bal_rank_mask -1 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:qdb_cluster leader: 24776 members: 24718,24730,24734,24752,24776 2024-09-21T01:21:49.718 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b{0:24776} state up:active seq 5 addr [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e{0:24707} state up:standby-replay seq 1 addr [v2:172.21.15.89:6834/3850699909,v1:172.21.15.89:6835/3850699909] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.g{1:24752} state up:active seq 20 addr [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f{1:24706} state up:standby-replay seq 1 addr [v2:172.21.15.98:6834/1764678372,v1:172.21.15.98:6835/1764678372] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.h{2:24734} state up:active seq 22 addr [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d{2:24695} state up:standby-replay seq 1 addr [v2:172.21.15.27:6836/392681445,v1:172.21.15.27:6837/392681445] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.i{3:24730} state up:active seq 22 addr [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.j{3:24686} state up:standby-replay seq 1 addr [v2:172.21.15.27:6834/2864798916,v1:172.21.15.27:6835/2864798916] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c{4:24718} state up:active seq 23 addr [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.k{4:24674} state up:standby-replay seq 1 addr [v2:172.21.15.89:6832/39744273,v1:172.21.15.89:6833/39744273] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:49.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a{-1:14721} state up:standby seq 1 addr [v2:172.21.15.27:6840/2646952373,v1:172.21.15.27:6841/2646952373] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.720 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.l{-1:24661} state up:standby seq 1 addr [v2:172.21.15.98:6832/2535514726,v1:172.21.15.98:6833/2535514726] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T01:21:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:21:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba240b6560 msgr2=0x7fba240b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba240b6560 0x7fba240b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba30009fc0 tx=0x7fba30005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 msgr2=0x7fba4019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 0x7fba4019cc90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba28005ab0 tx=0x7fba28003b90 comp rx=0 tx=0).stop 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 shutdown_connections 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fba240b6560 0x7fba240b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fba4010c010 0x7fba401a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba4010b6c0 0x7fba4019cc90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 --2- 172.21.15.27:0/120274959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba40105d80 0x7fba4019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.718+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 >> 172.21.15.27:0/120274959 conn(0x7fba40101810 msgr2=0x7fba40103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.719+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 shutdown_connections 2024-09-21T01:21:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:49.719+0000 7fba45000640 1 -- 172.21.15.27:0/120274959 wait complete. 2024-09-21T01:21:49.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:49 smithi098 ceph-mon[34372]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:21:49.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:49 smithi027 ceph-mon[30375]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:21:50.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:49 smithi089 ceph-mon[34486]: pgmap v382: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:21:50.529 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T01:21:50.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:50 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/120274959' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T01:21:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:50.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:50 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/120274959' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T01:21:50.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:50 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/120274959' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 -- 172.21.15.27:0/4289900714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1039b0 msgr2=0x7fde4c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 --2- 172.21.15.27:0/4289900714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1039b0 0x7fde4c103dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009900 tx=0x7fde3c02a410 comp rx=0 tx=0).stop 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 -- 172.21.15.27:0/4289900714 shutdown_connections 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 --2- 172.21.15.27:0/4289900714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 --2- 172.21.15.27:0/4289900714 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1092f0 0x7fde4c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 --2- 172.21.15.27:0/4289900714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1039b0 0x7fde4c103dc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 -- 172.21.15.27:0/4289900714 >> 172.21.15.27:0/4289900714 conn(0x7fde4c0ff440 msgr2=0x7fde4c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 -- 172.21.15.27:0/4289900714 shutdown_connections 2024-09-21T01:21:51.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.545+0000 7fde51a00640 1 -- 172.21.15.27:0/4289900714 wait complete. 2024-09-21T01:21:51.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.546+0000 7fde51a00640 1 Processor -- start 2024-09-21T01:21:51.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.546+0000 7fde51a00640 1 -- start start 2024-09-21T01:21:51.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1039b0 0x7fde4c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:51.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 0x7fde4c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:51.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:51.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c113900 con 0x7fde4c1092f0 2024-09-21T01:21:51.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c113780 con 0x7fde4c1039b0 2024-09-21T01:21:51.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c113a80 con 0x7fde4c109c40 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 0x7fde4c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde4be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1039b0 0x7fde4c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde43e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 0x7fde4c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49700/0 (socket says 172.21.15.27:49700) 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde51000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c1a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:38990/0 (socket says 172.21.15.27:38990) 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.547+0000 7fde43e00640 1 -- 172.21.15.27:0/2500211712 learned_addr learned my addr 172.21.15.27:0/2500211712 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde4be00640 1 -- 172.21.15.27:0/2500211712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 msgr2=0x7fde4c1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:51.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde4be00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde4be00640 1 -- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 msgr2=0x7fde4c19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde4be00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 0x7fde4c19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde4be00640 1 -- 172.21.15.27:0/2500211712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c1a2480 con 0x7fde4c1039b0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde43e00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 0x7fde4c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde51000640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde4be00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1039b0 0x7fde4c19c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02a930 tx=0x7fde3c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c03f030 con 0x7fde4c1039b0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c1a2710 con 0x7fde4c1039b0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde3c004490 con 0x7fde4c1039b0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.548+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c035070 con 0x7fde4c1039b0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.549+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1a8ed0 con 0x7fde4c1039b0 2024-09-21T01:21:51.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.549+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c103dc0 con 0x7fde4c1039b0 2024-09-21T01:21:51.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.552+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fde3c042030 con 0x7fde4c1039b0 2024-09-21T01:21:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.554+0000 7fde49600640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde300b6560 0x7fde300b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.554+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fde3c107d80 con 0x7fde4c1039b0 2024-09-21T01:21:51.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.554+0000 7fde43e00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde300b6560 0x7fde300b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:51.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.555+0000 7fde43e00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde300b6560 0x7fde300b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde4c108e90 tx=0x7fde34005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.556+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde3c0d2500 con 0x7fde4c1039b0 2024-09-21T01:21:51.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:51 smithi027 ceph-mon[30375]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-21T01:21:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.761+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fde4c19d640 con 0x7fde4c1039b0 2024-09-21T01:21:51.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.763+0000 7fde49600640 1 -- 172.21.15.27:0/2500211712 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+6042 (secure 0 0 0) 0x7fde3c0d7440 con 0x7fde4c1039b0 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 90 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:fsid fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-09-21T01:09:34.737222+0000 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2024-09-21T01:20:42.161651+0000 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:crush_version 27 2024-09-21T01:21:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:full_ratio 0.95 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:backfillfull_ratio 0.9 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:nearfull_ratio 0.85 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:require_min_compat_client luminous 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:min_compat_client luminous 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:require_osd_release squid 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:stretch_mode_enabled false 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:max_osd 12 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 up in weight 1 up_from 9 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6802/3167750643,v1:172.21.15.27:6803/3167750643] [v2:172.21.15.27:6804/3167750643,v1:172.21.15.27:6805/3167750643] exists,up ac7acdbc-3b48-4cdc-b04e-46d6930e274d 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 up in weight 1 up_from 14 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/1010396309,v1:172.21.15.89:6801/1010396309] [v2:172.21.15.89:6802/1010396309,v1:172.21.15.89:6803/1010396309] exists,up 5714e04c-89ef-47b5-9d28-8cc23530a45e 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 up in weight 1 up_from 19 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/2706735365,v1:172.21.15.98:6801/2706735365] [v2:172.21.15.98:6802/2706735365,v1:172.21.15.98:6803/2706735365] exists,up 3493a644-896e-4c12-bcf2-62a3aaa3f7d2 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6810/4289720168,v1:172.21.15.27:6811/4289720168] [v2:172.21.15.27:6812/4289720168,v1:172.21.15.27:6813/4289720168] exists,up ae443ec8-2955-493e-a422-395e3948b860 2024-09-21T01:21:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 up in weight 1 up_from 32 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/2496831965,v1:172.21.15.89:6809/2496831965] [v2:172.21.15.89:6810/2496831965,v1:172.21.15.89:6811/2496831965] exists,up 00c2670f-c063-4152-8f27-c437a83f1e03 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/3161787475,v1:172.21.15.98:6809/3161787475] [v2:172.21.15.98:6810/3161787475,v1:172.21.15.98:6811/3161787475] exists,up a8fa79ae-691d-44f3-b727-db027e158d64 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6818/1151323802,v1:172.21.15.27:6819/1151323802] [v2:172.21.15.27:6820/1151323802,v1:172.21.15.27:6821/1151323802] exists,up 17a579ce-a646-47ff-abcc-9d967c03d62c 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/2410726915,v1:172.21.15.89:6817/2410726915] [v2:172.21.15.89:6818/2410726915,v1:172.21.15.89:6819/2410726915] exists,up 3ce20c01-8543-4ca7-8986-e2cee9ae1f14 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.8 up in weight 1 up_from 54 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/1001086058,v1:172.21.15.98:6817/1001086058] [v2:172.21.15.98:6818/1001086058,v1:172.21.15.98:6819/1001086058] exists,up 86419495-74dd-48c5-a17f-c4c4c7411658 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.9 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6826/1133805797,v1:172.21.15.27:6827/1133805797] [v2:172.21.15.27:6828/1133805797,v1:172.21.15.27:6829/1133805797] exists,up 7d3a6679-2483-45b0-99b5-55c8668cb2b6 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.10 up in weight 1 up_from 65 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6824/4138240961,v1:172.21.15.89:6825/4138240961] [v2:172.21.15.89:6826/4138240961,v1:172.21.15.89:6827/4138240961] exists,up 1978e43b-cf1a-4a0c-a1bf-11a8ce290970 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:osd.11 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/165111690,v1:172.21.15.98:6825/165111690] [v2:172.21.15.98:6826/165111690,v1:172.21.15.98:6827/165111690] exists,up 0eb4e911-7e6f-4cf0-b60c-983aed5f5b06 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.5 [8,11] 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.c [2,4] 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.f [8,11] 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 2.14 [2,4] 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.2 [3,9,5,7] 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.3 [4,9] 2024-09-21T01:21:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1d [4,6] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.1e [3,9] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 3.21 [4,8] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.6 [5,8] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.c [9,10] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.17 [9,8] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:pg_upmap_items 4.1f [9,8] 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/525441395 expires 2024-09-22T01:11:40.306879+0000 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/3673025816 expires 2024-09-22T01:10:00.491967+0000 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/3673025816 expires 2024-09-22T01:10:00.491967+0000 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3489401183 expires 2024-09-22T01:10:00.491967+0000 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/2219512661 expires 2024-09-22T01:10:00.491967+0000 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1438694632 expires 2024-09-22T01:10:00.491967+0000 2024-09-21T01:21:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/3049262375 expires 2024-09-22T01:10:32.417709+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/3134270458 expires 2024-09-22T01:10:32.417709+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/3049262375 expires 2024-09-22T01:10:32.417709+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/365392204 expires 2024-09-22T01:10:32.417709+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1444171027 expires 2024-09-22T01:11:40.306879+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/492430082 expires 2024-09-22T01:10:32.417709+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6800/1523735126 expires 2024-09-22T01:11:40.306879+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:0/1528176571 expires 2024-09-22T01:11:40.306879+0000 2024-09-21T01:21:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:blocklist 172.21.15.27:6801/1523735126 expires 2024-09-22T01:11:40.306879+0000 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde300b6560 msgr2=0x7fde300b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde300b6560 0x7fde300b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde4c108e90 tx=0x7fde34005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1039b0 msgr2=0x7fde4c19c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1039b0 0x7fde4c19c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02a930 tx=0x7fde3c031a20 comp rx=0 tx=0).stop 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 shutdown_connections 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde300b6560 0x7fde300b8a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde4c109c40 0x7fde4c1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1092f0 0x7fde4c19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 --2- 172.21.15.27:0/2500211712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde4c1039b0 0x7fde4c19c510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.766+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 >> 172.21.15.27:0/2500211712 conn(0x7fde4c0ff440 msgr2=0x7fde4c10e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.767+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 shutdown_connections 2024-09-21T01:21:51.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:51.767+0000 7fde51a00640 1 -- 172.21.15.27:0/2500211712 wait complete. 2024-09-21T01:21:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:51 smithi098 ceph-mon[34372]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-21T01:21:51.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:51 smithi089 ceph-mon[34486]: pgmap v383: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-21T01:21:52.718 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T01:21:52.729 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:21:52.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:52 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2500211712' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T01:21:52.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:52 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2500211712' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T01:21:52.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:52 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2500211712' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T01:21:53.115 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 -- 172.21.15.27:0/1997117979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d0105110 msgr2=0x7fa9d010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/1997117979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d0105110 0x7fa9d010d660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009900 tx=0x7fa9bc02a410 comp rx=0 tx=0).stop 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 -- 172.21.15.27:0/1997117979 shutdown_connections 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/1997117979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d010dc20 0x7fa9d0110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/1997117979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d0105110 0x7fa9d010d660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/1997117979 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d01047c0 0x7fa9d0104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 -- 172.21.15.27:0/1997117979 >> 172.21.15.27:0/1997117979 conn(0x7fa9d00fe2c0 msgr2=0x7fa9d0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.776+0000 7fa9d5e00640 1 -- 172.21.15.27:0/1997117979 shutdown_connections 2024-09-21T01:21:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.777+0000 7fa9d5e00640 1 -- 172.21.15.27:0/1997117979 wait complete. 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.777+0000 7fa9d5e00640 1 Processor -- start 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 -- start start 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 0x7fa9d019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d0105110 0x7fa9d019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d010dc20 0x7fa9d019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0113880 con 0x7fa9d010dc20 2024-09-21T01:21:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0113700 con 0x7fa9d0105110 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d5e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0113a00 con 0x7fa9d01047c0 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d4a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 0x7fa9d019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9cfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d0105110 0x7fa9d019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d4a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 0x7fa9d019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:39014/0 (socket says 172.21.15.27:39014) 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9d4a00640 1 -- 172.21.15.27:0/3381937718 learned_addr learned my addr 172.21.15.27:0/3381937718 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.778+0000 7fa9cfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d0105110 0x7fa9d019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:41172/0 (socket says 172.21.15.27:41172) 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d5400640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d010dc20 0x7fa9d019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d4a00640 1 -- 172.21.15.27:0/3381937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d0105110 msgr2=0x7fa9d019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d4a00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d0105110 0x7fa9d019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d4a00640 1 -- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d010dc20 msgr2=0x7fa9d019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d4a00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d010dc20 0x7fa9d019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d4a00640 1 -- 172.21.15.27:0/3381937718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d01a0220 con 0x7fa9d01047c0 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.779+0000 7fa9d4a00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 0x7fa9d019a2b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400e9f0 tx=0x7fa9c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.780+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400cd80 con 0x7fa9d01047c0 2024-09-21T01:21:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.780+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c40108e0 con 0x7fa9d01047c0 2024-09-21T01:21:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.780+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d01a0510 con 0x7fa9d01047c0 2024-09-21T01:21:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.780+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4018980 con 0x7fa9d01047c0 2024-09-21T01:21:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.780+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d01a6c70 con 0x7fa9d01047c0 2024-09-21T01:21:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.781+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d0104bd0 con 0x7fa9d01047c0 2024-09-21T01:21:53.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.783+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fa9c4010430 con 0x7fa9d01047c0 2024-09-21T01:21:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.785+0000 7fa9cd600640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9b80b6560 0x7fa9b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.785+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa9c40e7fe0 con 0x7fa9d01047c0 2024-09-21T01:21:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.785+0000 7fa9cfe00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9b80b6560 0x7fa9b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.786+0000 7fa9cfe00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9b80b6560 0x7fa9b80b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009230 tx=0x7fa9bc031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:53.786+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9c40b2780 con 0x7fa9d01047c0 2024-09-21T01:21:53.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:53 smithi098 ceph-mon[34372]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:21:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:53 smithi027 ceph-mon[30375]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:21:53.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:53 smithi089 ceph-mon[34486]: pgmap v384: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:21:54.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.025+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9d019b3e0 con 0x7fa9d01047c0 2024-09-21T01:21:54.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.026+0000 7fa9cd600640 1 -- 172.21.15.27:0/3381937718 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7fa9c40b76c0 con 0x7fa9d01047c0 2024-09-21T01:21:54.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:54.027 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T01:20:08:397428+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:20:07.570104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24718,24730,24734,24752,24776]},"id":1}]} 2024-09-21T01:21:54.027 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:21:54.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.028+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9b80b6560 msgr2=0x7fa9b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:54.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.028+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9b80b6560 0x7fa9b80b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009230 tx=0x7fa9bc031400 comp rx=0 tx=0).stop 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.028+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 msgr2=0x7fa9d019a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.028+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 0x7fa9d019a2b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400e9f0 tx=0x7fa9c400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 shutdown_connections 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa9b80b6560 0x7fa9b80b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9d010dc20 0x7fa9d019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa9d0105110 0x7fa9d019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 --2- 172.21.15.27:0/3381937718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9d01047c0 0x7fa9d019a2b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 >> 172.21.15.27:0/3381937718 conn(0x7fa9d00fe2c0 msgr2=0x7fa9d00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 shutdown_connections 2024-09-21T01:21:54.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:54.029+0000 7fa9d5e00640 1 -- 172.21.15.27:0/3381937718 wait complete. 2024-09-21T01:21:54.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:54 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3381937718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:54.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:54 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3381937718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:54.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:21:54.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:54 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3381937718' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:55.317 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:55.805 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:55 smithi027 ceph-mon[30375]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T01:21:55.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:55 smithi098 ceph-mon[34372]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T01:21:55.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 -- 172.21.15.27:0/1501993130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c81039b0 msgr2=0x7f10c8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:55.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 --2- 172.21.15.27:0/1501993130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c81039b0 0x7f10c8103dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009900 tx=0x7f10b802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 -- 172.21.15.27:0/1501993130 shutdown_connections 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 --2- 172.21.15.27:0/1501993130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c8109c40 0x7f10c8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 --2- 172.21.15.27:0/1501993130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81092f0 0x7f10c8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 --2- 172.21.15.27:0/1501993130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c81039b0 0x7f10c8103dc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.987+0000 7f10cca00640 1 -- 172.21.15.27:0/1501993130 >> 172.21.15.27:0/1501993130 conn(0x7f10c80ff440 msgr2=0x7f10c8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.988+0000 7f10cca00640 1 -- 172.21.15.27:0/1501993130 shutdown_connections 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.988+0000 7f10cca00640 1 -- 172.21.15.27:0/1501993130 wait complete. 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 Processor -- start 2024-09-21T01:21:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 -- start start 2024-09-21T01:21:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 0x7f10c819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c81092f0 0x7f10c819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c8109c40 0x7f10c819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8113950 con 0x7f10c8109c40 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81137d0 con 0x7f10c81039b0 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.989+0000 7f10cca00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8113ad0 con 0x7f10c81092f0 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 0x7f10c819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c8109c40 0x7f10c819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 0x7f10c819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35580/0 (socket says 172.21.15.27:35580) 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 -- 172.21.15.27:0/3423725875 learned_addr learned my addr 172.21.15.27:0/3423725875 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c6a00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c81092f0 0x7f10c819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 -- 172.21.15.27:0/3423725875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c81092f0 msgr2=0x7f10c819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c81092f0 0x7f10c819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 -- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c8109c40 msgr2=0x7f10c819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c8109c40 0x7f10c819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 -- 172.21.15.27:0/3423725875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c81a0290 con 0x7f10c81039b0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.990+0000 7f10c7400640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 0x7f10c819a320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10b8005ab0 tx=0x7f10b8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.991+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b803f070 con 0x7f10c81039b0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.991+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c81a0520 con 0x7f10c81039b0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.991+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b8031b60 con 0x7f10c81039b0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.991+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8035030 con 0x7f10c81039b0 2024-09-21T01:21:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.991+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c80755f0 con 0x7f10c81039b0 2024-09-21T01:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.992+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c8104c30 con 0x7f10c81039b0 2024-09-21T01:21:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.995+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f10b8042070 con 0x7f10c81039b0 2024-09-21T01:21:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.996+0000 7f10bfe00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10ac0b6560 0x7f10ac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.996+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f10b8108750 con 0x7f10c81039b0 2024-09-21T01:21:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.996+0000 7f10c6a00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10ac0b6560 0x7f10ac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.997+0000 7f10c6a00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10ac0b6560 0x7f10ac0b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f10b40095f0 tx=0x7f10b4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:55.998+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10b80d2ed0 con 0x7f10c81039b0 2024-09-21T01:21:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:55 smithi089 ceph-mon[34486]: pgmap v385: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T01:21:56.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.237+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c8103dc0 con 0x7f10c81039b0 2024-09-21T01:21:56.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.238+0000 7f10bfe00640 1 -- 172.21.15.27:0/3423725875 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11960 (secure 0 0 0) 0x7f10b80d7e10 con 0x7f10c81039b0 2024-09-21T01:21:56.239 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:56.239 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"btime":"2024-09-21T01:20:08:397428+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:20:07.570104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24718,24730,24734,24752,24776]},"id":1}]} 2024-09-21T01:21:56.240 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10ac0b6560 msgr2=0x7f10ac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10ac0b6560 0x7f10ac0b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f10b40095f0 tx=0x7f10b4005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 msgr2=0x7f10c819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 0x7f10c819a320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10b8005ab0 tx=0x7f10b8003c10 comp rx=0 tx=0).stop 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 shutdown_connections 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f10ac0b6560 0x7f10ac0b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10c8109c40 0x7f10c819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10c81092f0 0x7f10c819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 --2- 172.21.15.27:0/3423725875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10c81039b0 0x7f10c819a320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:56.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 >> 172.21.15.27:0/3423725875 conn(0x7f10c80ff440 msgr2=0x7f10c8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:56.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 shutdown_connections 2024-09-21T01:21:56.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:56.241+0000 7f10cca00640 1 -- 172.21.15.27:0/3423725875 wait complete. 2024-09-21T01:21:56.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:56 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3423725875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:56.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:56 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3423725875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:57.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:56 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3423725875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:21:57.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:21:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:57.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:57 smithi098 ceph-mon[34372]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T01:21:57.906 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:57 smithi027 ceph-mon[30375]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T01:21:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:57 smithi089 ceph-mon[34486]: pgmap v386: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 -- 172.21.15.27:0/3290182585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8109c40 msgr2=0x7ff1f8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 --2- 172.21.15.27:0/3290182585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8109c40 0x7ff1f8110470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009ca0 tx=0x7ff1f002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 -- 172.21.15.27:0/3290182585 shutdown_connections 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 --2- 172.21.15.27:0/3290182585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f8109c40 0x7ff1f8110470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 --2- 172.21.15.27:0/3290182585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 --2- 172.21.15.27:0/3290182585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f81039b0 0x7ff1f8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 -- 172.21.15.27:0/3290182585 >> 172.21.15.27:0/3290182585 conn(0x7ff1f80ff400 msgr2=0x7ff1f8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 -- 172.21.15.27:0/3290182585 shutdown_connections 2024-09-21T01:21:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.257+0000 7ff1fce00640 1 -- 172.21.15.27:0/3290182585 wait complete. 2024-09-21T01:21:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.258+0000 7ff1fce00640 1 Processor -- start 2024-09-21T01:21:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.258+0000 7ff1fce00640 1 -- start start 2024-09-21T01:21:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1fce00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f81039b0 0x7ff1f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1fce00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1fce00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8109c40 0x7ff1f8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1fce00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113900 con 0x7ff1f81039b0 2024-09-21T01:21:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1fce00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113780 con 0x7ff1f81092f0 2024-09-21T01:21:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1fce00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113a80 con 0x7ff1f8109c40 2024-09-21T01:21:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1f7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f81039b0 0x7ff1f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1f6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35598/0 (socket says 172.21.15.27:35598) 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2474219918 learned_addr learned my addr 172.21.15.27:0/2474219918 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.259+0000 7ff1f7e00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8109c40 0x7ff1f8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2474219918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8109c40 msgr2=0x7ff1f8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8109c40 0x7ff1f8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f81039b0 msgr2=0x7ff1f8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f81039b0 0x7ff1f8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1f6a00640 1 -- 172.21.15.27:0/2474219918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a8550 con 0x7ff1f81092f0 2024-09-21T01:21:58.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1f6a00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8075c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e9f0 tx=0x7ff1e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400ccc0 con 0x7ff1f81092f0 2024-09-21T01:21:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a8840 con 0x7ff1f81092f0 2024-09-21T01:21:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e40108e0 con 0x7ff1f81092f0 2024-09-21T01:21:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.260+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4018860 con 0x7ff1f81092f0 2024-09-21T01:21:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.261+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a8dd0 con 0x7ff1f81092f0 2024-09-21T01:21:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.262+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8103dc0 con 0x7ff1f81092f0 2024-09-21T01:21:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.264+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7ff1e4010430 con 0x7ff1f81092f0 2024-09-21T01:21:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.266+0000 7ff1efe00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1dc0b6560 0x7ff1dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:21:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.266+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff1e40e7fd0 con 0x7ff1f81092f0 2024-09-21T01:21:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.266+0000 7ff1f7400640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1dc0b6560 0x7ff1dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:21:58.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.267+0000 7ff1f7400640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1dc0b6560 0x7ff1dc0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8077160 tx=0x7ff1e8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:21:58.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.268+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff1e40b2750 con 0x7ff1f81092f0 2024-09-21T01:21:58.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.470+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1f802e640 con 0x7ff1f81092f0 2024-09-21T01:21:58.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.472+0000 7ff1efe00640 1 -- 172.21.15.27:0/2474219918 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25150 (secure 0 0 0) 0x7ff1e40b7690 con 0x7ff1f81092f0 2024-09-21T01:21:58.473 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:21:58.473 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:20:42.161651+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:21:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1dc0b6560 msgr2=0x7ff1dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1dc0b6560 0x7ff1dc0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8077160 tx=0x7ff1e8005f90 comp rx=0 tx=0).stop 2024-09-21T01:21:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 msgr2=0x7ff1f8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8075c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e9f0 tx=0x7ff1e400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 shutdown_connections 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff1dc0b6560 0x7ff1dc0b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.474+0000 7ff1fce00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f8109c40 0x7ff1f8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.475+0000 7ff1fce00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f81092f0 0x7ff1f8075c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.475+0000 7ff1fce00640 1 --2- 172.21.15.27:0/2474219918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1f81039b0 0x7ff1f8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.475+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 >> 172.21.15.27:0/2474219918 conn(0x7ff1f80ff400 msgr2=0x7ff1f8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.475+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 shutdown_connections 2024-09-21T01:21:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:21:58.475+0000 7ff1fce00640 1 -- 172.21.15.27:0/2474219918 wait complete. 2024-09-21T01:21:58.820 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:58 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2474219918' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:21:58.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:58 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2474219918' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:21:59.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:58 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2474219918' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:21:59.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T01:19:07.935504+0000', 'modified': '2024-09-21T01:20:07.570104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24776, 'mds_1': 24752, 'mds_2': 24734, 'mds_3': 24730, 'mds_4': 24718}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6833/39744273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6832', 'nonce': 39744273}, {'type': 'v1', 'addr': '172.21.15.89:6833', 'nonce': 39744273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24686': {'gid': 24686, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6835/2864798916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 2864798916}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2864798916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6837/392681445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6836', 'nonce': 392681445}, {'type': 'v1', 'addr': '172.21.15.27:6837', 'nonce': 392681445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24706': {'gid': 24706, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1764678372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1764678372}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1764678372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6835/3850699909', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 3850699909}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 3850699909}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24718': {'gid': 24718, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.98:6837/1081631404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1081631404}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1081631404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24730': {'gid': 24730, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.98:6839/3799755243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3799755243}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3799755243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24734': {'gid': 24734, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.89:6837/3556269004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3556269004}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3556269004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24752': {'gid': 24752, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.27:6839/3115497172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3115497172}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3115497172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24776': {'gid': 24776, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4156411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4156411896}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4156411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24776, 'qdb_cluster': [24718, 24730, 24734, 24752, 24776]} 2024-09-21T01:21:59.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T01:21:59.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T01:21:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:21:59.884 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:21:59 smithi027 ceph-mon[30375]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:21:59.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:21:59 smithi098 ceph-mon[34372]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:21:59 smithi089 ceph-mon[34486]: pgmap v387: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 -- 172.21.15.27:0/1865051657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f901094a0 msgr2=0x7f3f901098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 --2- 172.21.15.27:0/1865051657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f901094a0 0x7f3f901098b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009900 tx=0x7f3f7c02a410 comp rx=0 tx=0).stop 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 -- 172.21.15.27:0/1865051657 shutdown_connections 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 --2- 172.21.15.27:0/1865051657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f90109df0 0x7f3f90110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 --2- 172.21.15.27:0/1865051657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f901094a0 0x7f3f901098b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 --2- 172.21.15.27:0/1865051657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f90103b60 0x7f3f90103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.448+0000 7f3f98600640 1 -- 172.21.15.27:0/1865051657 >> 172.21.15.27:0/1865051657 conn(0x7f3f9007b850 msgr2=0x7f3f9007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.449+0000 7f3f98600640 1 -- 172.21.15.27:0/1865051657 shutdown_connections 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.449+0000 7f3f98600640 1 -- 172.21.15.27:0/1865051657 wait complete. 2024-09-21T01:22:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.450+0000 7f3f98600640 1 Processor -- start 2024-09-21T01:22:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.450+0000 7f3f98600640 1 -- start start 2024-09-21T01:22:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f98600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 0x7f3f9019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f98600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f901094a0 0x7f3f9019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f98600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f90109df0 0x7f3f9019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f98600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90113ab0 con 0x7f3f90103b60 2024-09-21T01:22:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f98600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90113930 con 0x7f3f901094a0 2024-09-21T01:22:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f98600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90113c30 con 0x7f3f90109df0 2024-09-21T01:22:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f95c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 0x7f3f9019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f95c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 0x7f3f9019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37084/0 (socket says 172.21.15.27:37084) 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f95c00640 1 -- 172.21.15.27:0/2451328360 learned_addr learned my addr 172.21.15.27:0/2451328360 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f95200640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f901094a0 0x7f3f9019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.451+0000 7f3f96600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f90109df0 0x7f3f9019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f95c00640 1 -- 172.21.15.27:0/2451328360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f90109df0 msgr2=0x7f3f9019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f95c00640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f90109df0 0x7f3f9019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f95c00640 1 -- 172.21.15.27:0/2451328360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f901094a0 msgr2=0x7f3f9019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f95c00640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f901094a0 0x7f3f9019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f95c00640 1 -- 172.21.15.27:0/2451328360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f901a03b0 con 0x7f3f90103b60 2024-09-21T01:22:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f95c00640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 0x7f3f9019a440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400eb90 tx=0x7f3f8400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:00.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f8e000640 1 -- 172.21.15.27:0/2451328360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84004d20 con 0x7f3f90103b60 2024-09-21T01:22:00.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f8e000640 1 -- 172.21.15.27:0/2451328360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8400ee90 con 0x7f3f90103b60 2024-09-21T01:22:00.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f901a06a0 con 0x7f3f90103b60 2024-09-21T01:22:00.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.452+0000 7f3f8e000640 1 -- 172.21.15.27:0/2451328360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84010670 con 0x7f3f90103b60 2024-09-21T01:22:00.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.453+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f901a6e00 con 0x7f3f90103b60 2024-09-21T01:22:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.456+0000 7f3f9f266740 1 Processor -- start 2024-09-21T01:22:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.456+0000 7f3f9f266740 1 -- start start 2024-09-21T01:22:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f8e000640 1 -- 172.21.15.27:0/2451328360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f3f84005020 con 0x7f3f90103b60 2024-09-21T01:22:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 0x557026d4a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55702697b0a0 0x557026d4a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d4af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f9f266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557026cc1d50 con 0x5570269a3f60 2024-09-21T01:22:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f9f266740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557026cb1180 con 0x557026983280 2024-09-21T01:22:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f9f266740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557026cb4f90 con 0x55702697b0a0 2024-09-21T01:22:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f96600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 0x557026d4a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f96600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 0x557026d4a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37096/0 (socket says 172.21.15.27:37096) 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f95c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55702697b0a0 0x557026d4a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f96600640 1 -- 172.21.15.27:0/2459846704 learned_addr learned my addr 172.21.15.27:0/2459846704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.457+0000 7f3f95200640 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d4af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.458+0000 7f3f96600640 1 -- 172.21.15.27:0/2459846704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55702697b0a0 msgr2=0x557026d4a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.458+0000 7f3f96600640 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55702697b0a0 0x557026d4a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.458+0000 7f3f96600640 1 -- 172.21.15.27:0/2459846704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 msgr2=0x557026d4af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.458+0000 7f3f96600640 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d4af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.458+0000 7f3f96600640 1 -- 172.21.15.27:0/2459846704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557026c7ba70 con 0x5570269a3f60 2024-09-21T01:22:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.458+0000 7f3f96600640 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 0x557026d4a490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ca0 tx=0x7f3f88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e65f1b3745fc59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.459+0000 7f3f82a00640 1 -- 172.21.15.27:0/2459846704 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8801e070 con 0x5570269a3f60 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.459+0000 7f3f82a00640 1 -- 172.21.15.27:0/2459846704 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f88009e50 con 0x5570269a3f60 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.459+0000 7f3f82a00640 1 -- 172.21.15.27:0/2459846704 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800e9b0 con 0x5570269a3f60 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.459+0000 7f3f8e000640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f780b65e0 0x7f3f780b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.459+0000 7f3f8e000640 1 -- 172.21.15.27:0/2451328360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3f840e7e50 con 0x7f3f90103b60 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.459+0000 7f3f95c00640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f780b65e0 0x7f3f780b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 -- 172.21.15.27:0/2459846704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 msgr2=0x557026d4a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 0x557026d4a490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ca0 tx=0x7f3f88011810 comp rx=0 tx=0).stop 2024-09-21T01:22:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 -- 172.21.15.27:0/2459846704 shutdown_connections 2024-09-21T01:22:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55702697b0a0 0x557026d4a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2459846704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5570269a3f60 0x557026d4a490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 -- 172.21.15.27:0/2459846704 >> 172.21.15.27:0/2459846704 conn(0x557026998350 msgr2=0x557026d48380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f95c00640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f780b65e0 0x7f3f780b8af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3f840aa0c0 tx=0x7f3f840eb890 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 -- 172.21.15.27:0/2459846704 shutdown_connections 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.460+0000 7f3f9f266740 1 -- 172.21.15.27:0/2459846704 wait complete. 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 -- start start 2024-09-21T01:22:00.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 Processor -- start 2024-09-21T01:22:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 0x557026d5a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557026983280 0x557026d5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d66300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557026b125f0 con 0x557026983280 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557026c4c460 con 0x55702697b0a0 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.461+0000 7f3f9f266740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557026d5cfa0 con 0x5570269a3f60 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f9f266740 10 client.0 _ll_register_callbacks cb 0x557026c5c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 0x557026d5a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f95200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557026983280 0x557026d5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 0x557026d5a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35638/0 (socket says 172.21.15.27:35638) 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 -- 172.21.15.27:0/79614254 learned_addr learned my addr 172.21.15.27:0/79614254 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d66300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 -- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 msgr2=0x557026d66300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d66300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 -- 172.21.15.27:0/79614254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557026983280 msgr2=0x557026d5a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557026983280 0x557026d5a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f96600640 1 -- 172.21.15.27:0/79614254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557026a0f540 con 0x55702697b0a0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.462+0000 7f3f95200640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557026983280 0x557026d5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f96600640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 0x557026d5a3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ab0 tx=0x7f3f8800e550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ba4ecce84872 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f77400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f77400640 1 -- 172.21.15.27:0/79614254 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8802e060 con 0x55702697b0a0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557026c4ea70 con 0x55702697b0a0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f77400640 1 -- 172.21.15.27:0/79614254 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3f88004460 con 0x55702697b0a0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f77400640 1 -- 172.21.15.27:0/79614254 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f88035050 con 0x55702697b0a0 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f9f266740 10 client.25370 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:00.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557026c696f0 con 0x55702697b0a0 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.463+0000 7f3f77400640 1 -- 172.21.15.27:0/79614254 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557026c4ea70 con 0x55702697b0a0 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f77400640 1 -- 172.21.15.27:0/79614254 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3f880049f0 con 0x55702697b0a0 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f9f266740 10 client.25370 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f9f266740 10 client.25370 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f9f266740 20 client.25370 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f9f266740 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5570269628e0 0x557026d6c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f9f266740 4 client.25370 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.464+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 --> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557026cc3ca0 con 0x5570269628e0 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.465+0000 7f3f95200640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5570269628e0 0x557026d6c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.465+0000 7f3f95200640 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5570269628e0 0x557026d6c5c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19792d125f17daa1 server_cookie=5b98215053073752 in_seq=0 out_seq=0 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.465+0000 7f3f77400640 10 client.25370 ms_handle_connect on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f77400640 1 -- 172.21.15.27:0/79614254 <== mds.4 v2:172.21.15.98:6836/1081631404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557026cc3ca0 con 0x5570269628e0 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f77400640 10 client.25370 handle_command_reply: tid=0 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 client.25370 shutdown 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 msgr2=0x557026d5a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 0x557026d5a3d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ab0 tx=0x7f3f8800e550 comp rx=0 tx=0).stop 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 shutdown_connections 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x5570269628e0 0x557026d6c5c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d66300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557026983280 0x557026d5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 --2- 172.21.15.27:0/79614254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55702697b0a0 0x557026d5a3d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 >> 172.21.15.27:0/79614254 conn(0x557026998350 msgr2=0x557026d46920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:00.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.466+0000 7f3f77400640 0 client.25370 ms_handle_reset on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:00.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.467+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 shutdown_connections 2024-09-21T01:22:00.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.467+0000 7f3f9f266740 1 -- 172.21.15.27:0/79614254 wait complete. 2024-09-21T01:22:00.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.467+0000 7f3f9f266740 20 client.25370 trim_cache size 0 max 16384 2024-09-21T01:22:00.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.486+0000 7f3f9f266740 1 Processor -- start 2024-09-21T01:22:00.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.486+0000 7f3f9f266740 1 -- start start 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d66270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d667b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f9f266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557026d6a900 con 0x55702697b0a0 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f9f266740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557026d6afd0 con 0x557026983280 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f9f266740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557026d6b6a0 con 0x5570269a3f60 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f96600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f96600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37114/0 (socket says 172.21.15.27:37114) 2024-09-21T01:22:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f96600640 1 -- 172.21.15.27:0/2720777177 learned_addr learned my addr 172.21.15.27:0/2720777177 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f95200640 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d66270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.487+0000 7f3f96600640 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d667b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f96600640 1 -- 172.21.15.27:0/2720777177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 msgr2=0x557026d667b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f96600640 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d667b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f96600640 1 -- 172.21.15.27:0/2720777177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 msgr2=0x557026d66270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f96600640 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d66270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f96600640 1 -- 172.21.15.27:0/2720777177 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557026d5b490 con 0x55702697b0a0 2024-09-21T01:22:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f95200640 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d66270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f96600640 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6b150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f88006010 tx=0x7f3f88005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d263d484e69ee5be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f82000640 1 -- 172.21.15.27:0/2720777177 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8802e070 con 0x55702697b0a0 2024-09-21T01:22:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f82000640 1 -- 172.21.15.27:0/2720777177 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f88003e60 con 0x55702697b0a0 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.488+0000 7f3f82000640 1 -- 172.21.15.27:0/2720777177 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8801e070 con 0x55702697b0a0 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 -- 172.21.15.27:0/2720777177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 msgr2=0x557026d6b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6b150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f88006010 tx=0x7f3f88005970 comp rx=0 tx=0).stop 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 -- 172.21.15.27:0/2720777177 shutdown_connections 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d667b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d66270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 --2- 172.21.15.27:0/2720777177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6b150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 -- 172.21.15.27:0/2720777177 >> 172.21.15.27:0/2720777177 conn(0x557026998350 msgr2=0x557026d55420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 -- 172.21.15.27:0/2720777177 shutdown_connections 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.489+0000 7f3f9f266740 1 -- 172.21.15.27:0/2720777177 wait complete. 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 -- start start 2024-09-21T01:22:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 Processor -- start 2024-09-21T01:22:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d73420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x557026d6c440 con 0x55702697b0a0 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557026d6bd70 con 0x557026983280 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.490+0000 7f3f9f266740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557026d6cb10 con 0x5570269a3f60 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f9f266740 10 client.0 _ll_register_callbacks cb 0x7f3f88009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37126/0 (socket says 172.21.15.27:37126) 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 -- 172.21.15.27:0/817106326 learned_addr learned my addr 172.21.15.27:0/817106326 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f95200640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d73420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 -- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 msgr2=0x557026d73420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d73420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 -- 172.21.15.27:0/817106326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 msgr2=0x557026d6e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d6e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.491+0000 7f3f96600640 1 -- 172.21.15.27:0/817106326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557026d73a60 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f96600640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6dee0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e9b0 tx=0x7f3f88000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcb44bdecf06d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f77400640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f77400640 1 -- 172.21.15.27:0/817106326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8801e070 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f88003e60 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f77400640 1 -- 172.21.15.27:0/817106326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3f8800ea10 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f77400640 1 -- 172.21.15.27:0/817106326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f8802e040 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f9f266740 10 client.15333 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.492+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557026d74e40 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f77400640 1 -- 172.21.15.27:0/817106326 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f88003e60 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f77400640 1 -- 172.21.15.27:0/817106326 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3f88004710 con 0x55702697b0a0 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f9f266740 10 client.15333 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f9f266740 10 client.15333 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f9f266740 20 client.15333 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f9f266740 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x557026937c90 0x557026dda5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f9f266740 4 client.15333 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 --> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] -- command(tid 0: {"prefix": "status"}) -- 0x557026cbf750 con 0x557026937c90 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.493+0000 7f3f95200640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x557026937c90 0x557026dda5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f95200640 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x557026937c90 0x557026dda5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=665418b21cf3eaa9 server_cookie=2186b51b1f9dd07e in_seq=0 out_seq=0 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f77400640 10 client.15333 ms_handle_connect on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f77400640 1 -- 172.21.15.27:0/817106326 <== mds.4 v2:172.21.15.98:6836/1081631404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557026cbf750 con 0x557026937c90 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f77400640 10 client.15333 handle_command_reply: tid=0 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 client.15333 shutdown 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 msgr2=0x557026d6dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6dee0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800e9b0 tx=0x7f3f88000bc0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 shutdown_connections 2024-09-21T01:22:00.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x557026937c90 0x557026dda5f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5570269a3f60 0x557026d73420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557026983280 0x557026d6e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 --2- 172.21.15.27:0/817106326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55702697b0a0 0x557026d6dee0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 >> 172.21.15.27:0/817106326 conn(0x557026998350 msgr2=0x557026d5b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.494+0000 7f3f77400640 0 client.15333 ms_handle_reset on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.495+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 shutdown_connections 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.495+0000 7f3f9f266740 1 -- 172.21.15.27:0/817106326 wait complete. 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.495+0000 7f3f9f266740 20 client.15333 trim_cache size 0 max 16384 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-09-21T01:22:00.499 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24718, 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 116.12100720700001, 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 202.440118466 2024-09-21T01:22:00.500 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.495+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f780b65e0 msgr2=0x7f3f780b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.495+0000 7f3f98600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f780b65e0 0x7f3f780b8af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3f840aa0c0 tx=0x7f3f840eb890 comp rx=0 tx=0).stop 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 msgr2=0x7f3f9019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 0x7f3f9019a440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400eb90 tx=0x7f3f8400c6a0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 shutdown_connections 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f780b65e0 0x7f3f780b8af0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f90109df0 0x7f3f9019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f901094a0 0x7f3f9019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 --2- 172.21.15.27:0/2451328360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f90103b60 0x7f3f9019a440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 >> 172.21.15.27:0/2451328360 conn(0x7f3f9007b850 msgr2=0x7f3f901054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 shutdown_connections 2024-09-21T01:22:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:00.496+0000 7f3f98600640 1 -- 172.21.15.27:0/2451328360 wait complete. 2024-09-21T01:22:01.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T01:22:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:01.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:01 smithi027 ceph-mon[30375]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:22:01.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:01 smithi098 ceph-mon[34372]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:22:02.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:01 smithi089 ceph-mon[34486]: pgmap v388: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.348+0000 7f4d7cc00640 1 -- 172.21.15.27:0/2611618030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810be40 msgr2=0x7f4d78112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.348+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/2611618030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810be40 0x7f4d78112670 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009ca0 tx=0x7f4d7002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 -- 172.21.15.27:0/2611618030 shutdown_connections 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/2611618030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810be40 0x7f4d78112670 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/2611618030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d7810b4f0 0x7f4d7810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/2611618030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d78105bb0 0x7f4d78105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 -- 172.21.15.27:0/2611618030 >> 172.21.15.27:0/2611618030 conn(0x7f4d78101680 msgr2=0x7f4d78103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 -- 172.21.15.27:0/2611618030 shutdown_connections 2024-09-21T01:22:02.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.349+0000 7f4d7cc00640 1 -- 172.21.15.27:0/2611618030 wait complete. 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 Processor -- start 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 -- start start 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d78105bb0 0x7f4d7819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 0x7f4d7819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7810be40 0x7f4d781a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78115b50 con 0x7f4d7810b4f0 2024-09-21T01:22:02.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781159d0 con 0x7f4d7810be40 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.350+0000 7f4d7cc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78115cd0 con 0x7f4d78105bb0 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 0x7f4d7819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 0x7f4d7819cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37138/0 (socket says 172.21.15.27:37138) 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d6fe00640 1 -- 172.21.15.27:0/3215421491 learned_addr learned my addr 172.21.15.27:0/3215421491 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77400640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d78105bb0 0x7f4d7819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77e00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7810be40 0x7f4d781a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77400640 1 -- 172.21.15.27:0/3215421491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7810be40 msgr2=0x7f4d781a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77400640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7810be40 0x7f4d781a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77400640 1 -- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 msgr2=0x7f4d7819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77400640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 0x7f4d7819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d77400640 1 -- 172.21.15.27:0/3215421491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d781a2510 con 0x7f4d78105bb0 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.351+0000 7f4d6fe00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 0x7f4d7819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:02.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.352+0000 7f4d77400640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d78105bb0 0x7f4d7819c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e9f0 tx=0x7f4d6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.352+0000 7f4d74c00640 1 -- 172.21.15.27:0/3215421491 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800ccc0 con 0x7f4d78105bb0 2024-09-21T01:22:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.352+0000 7f4d74c00640 1 -- 172.21.15.27:0/3215421491 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d680108e0 con 0x7f4d78105bb0 2024-09-21T01:22:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.352+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d781a2800 con 0x7f4d78105bb0 2024-09-21T01:22:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.352+0000 7f4d74c00640 1 -- 172.21.15.27:0/3215421491 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68018880 con 0x7f4d78105bb0 2024-09-21T01:22:02.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.352+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a8f60 con 0x7f4d78105bb0 2024-09-21T01:22:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.355+0000 7f4d84e95740 1 Processor -- start 2024-09-21T01:22:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.355+0000 7f4d84e95740 1 -- start start 2024-09-21T01:22:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e0783670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03b40a0 0x55f5e0783bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03bc280 0x55f5e07840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d84e95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5e06fad50 con 0x55f5e03dcf60 2024-09-21T01:22:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d84e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f5e06ea180 con 0x55f5e03b40a0 2024-09-21T01:22:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d84e95740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5e06edf90 con 0x55f5e03bc280 2024-09-21T01:22:02.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e0783670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e0783670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37150/0 (socket says 172.21.15.27:37150) 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d77e00640 1 -- 172.21.15.27:0/3864904258 learned_addr learned my addr 172.21.15.27:0/3864904258 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d74c00640 1 -- 172.21.15.27:0/3215421491 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f4d68010430 con 0x7f4d78105bb0 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.356+0000 7f4d6fe00640 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03bc280 0x55f5e07840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d77e00640 1 -- 172.21.15.27:0/3864904258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03bc280 msgr2=0x55f5e07840f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d77e00640 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03bc280 0x55f5e07840f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d77e00640 1 -- 172.21.15.27:0/3864904258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03b40a0 msgr2=0x55f5e0783bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d77e00640 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03b40a0 0x55f5e0783bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d77e00640 1 -- 172.21.15.27:0/3864904258 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e06b4a70 con 0x55f5e03dcf60 2024-09-21T01:22:02.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d77e00640 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e0783670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009ca0 tx=0x7f4d70009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459a27e200f97b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d67e00640 1 -- 172.21.15.27:0/3864904258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70031cf0 con 0x55f5e03dcf60 2024-09-21T01:22:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d67e00640 1 -- 172.21.15.27:0/3864904258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70036030 con 0x55f5e03dcf60 2024-09-21T01:22:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.357+0000 7f4d67e00640 1 -- 172.21.15.27:0/3864904258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70004280 con 0x55f5e03dcf60 2024-09-21T01:22:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.358+0000 7f4d84e95740 1 -- 172.21.15.27:0/3864904258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 msgr2=0x55f5e0783670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.358+0000 7f4d84e95740 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e0783670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009ca0 tx=0x7f4d70009430 comp rx=0 tx=0).stop 2024-09-21T01:22:02.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.358+0000 7f4d74c00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4d5c0b65e0 0x7f4d5c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 -- 172.21.15.27:0/3864904258 shutdown_connections 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03bc280 0x55f5e07840f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d74c00640 1 -- 172.21.15.27:0/3215421491 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4d680e7b20 con 0x7f4d78105bb0 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03b40a0 0x55f5e0783bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 --2- 172.21.15.27:0/3864904258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e0783670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 -- 172.21.15.27:0/3864904258 >> 172.21.15.27:0/3864904258 conn(0x55f5e03d1350 msgr2=0x55f5e0781560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d77e00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4d5c0b65e0 0x7f4d5c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 -- 172.21.15.27:0/3864904258 shutdown_connections 2024-09-21T01:22:02.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.359+0000 7f4d84e95740 1 -- 172.21.15.27:0/3864904258 wait complete. 2024-09-21T01:22:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 -- start start 2024-09-21T01:22:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d77e00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4d5c0b65e0 0x7f4d5c0b8af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4d7002ade0 tx=0x7f4d7002a8c0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 Processor -- start 2024-09-21T01:22:02.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e079af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5e054b5f0 con 0x55f5e03b40a0 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f5e0685460 con 0x55f5e03bc280 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.360+0000 7f4d84e95740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5e0791c70 con 0x55f5e03dcf60 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d84e95740 10 client.0 _ll_register_callbacks cb 0x55f5e0695210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e079af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e079af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:53260/0 (socket says 172.21.15.27:53260) 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 -- 172.21.15.27:0/2210535215 learned_addr learned my addr 172.21.15.27:0/2210535215 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77400640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d6fe00640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 -- 172.21.15.27:0/2210535215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 msgr2=0x55f5e078f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 -- 172.21.15.27:0/2210535215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 msgr2=0x55f5e078f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.361+0000 7f4d77e00640 1 -- 172.21.15.27:0/2210535215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e0448540 con 0x55f5e03dcf60 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d77e00640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e079af60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d70031d70 tx=0x7f4d70003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94fcc50669e5abbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d64c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d64c00640 1 -- 172.21.15.27:0/2210535215 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7002f820 con 0x55f5e03dcf60 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5e0687a70 con 0x55f5e03dcf60 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d64c00640 1 -- 172.21.15.27:0/2210535215 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4d7002f9c0 con 0x55f5e03dcf60 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d64c00640 1 -- 172.21.15.27:0/2210535215 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d70067050 con 0x55f5e03dcf60 2024-09-21T01:22:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d84e95740 10 client.25375 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.362+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5e06a26f0 con 0x55f5e03dcf60 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d64c00640 1 -- 172.21.15.27:0/2210535215 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f5e0687a70 con 0x55f5e03dcf60 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d64c00640 1 -- 172.21.15.27:0/2210535215 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4d7004e050 con 0x55f5e03dcf60 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d84e95740 10 client.25375 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d84e95740 10 client.25375 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d84e95740 20 client.25375 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d84e95740 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e039b8e0 0x55f5e07a1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d84e95740 4 client.25375 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.363+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 --> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5e06fcca0 con 0x55f5e039b8e0 2024-09-21T01:22:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.364+0000 7f4d77400640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e039b8e0 0x55f5e07a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.364+0000 7f4d77400640 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e039b8e0 0x55f5e07a1250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a805637dd37ab450 server_cookie=7ec77801b81380de in_seq=0 out_seq=0 2024-09-21T01:22:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.364+0000 7f4d64c00640 10 client.25375 ms_handle_connect on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d64c00640 1 -- 172.21.15.27:0/2210535215 <== mds.3 v2:172.21.15.98:6838/3799755243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5e06fcca0 con 0x55f5e039b8e0 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d64c00640 10 client.25375 handle_command_reply: tid=0 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 client.25375 shutdown 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 msgr2=0x55f5e079af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e079af60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d70031d70 tx=0x7f4d70003ce0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 shutdown_connections 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e039b8e0 0x55f5e07a1250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e079af60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d64c00640 0 client.25375 ms_handle_reset on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 --2- 172.21.15.27:0/2210535215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 >> 172.21.15.27:0/2210535215 conn(0x55f5e03d1350 msgr2=0x55f5e0786850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.365+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 shutdown_connections 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.366+0000 7f4d84e95740 1 -- 172.21.15.27:0/2210535215 wait complete. 2024-09-21T01:22:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.366+0000 7f4d84e95740 20 client.25375 trim_cache size 0 max 16384 2024-09-21T01:22:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 Processor -- start 2024-09-21T01:22:02.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 -- start start 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e078a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5e079f590 con 0x55f5e03b40a0 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f5e079fc60 con 0x55f5e03bc280 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d84e95740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5e07a0330 con 0x55f5e03dcf60 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.385+0000 7f4d77400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37172/0 (socket says 172.21.15.27:37172) 2024-09-21T01:22:02.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 -- 172.21.15.27:0/138093501 learned_addr learned my addr 172.21.15.27:0/138093501 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:02.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77e00640 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e078a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d6fe00640 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 -- 172.21.15.27:0/138093501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 msgr2=0x55f5e078a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e078a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 -- 172.21.15.27:0/138093501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 msgr2=0x55f5e078f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 -- 172.21.15.27:0/138093501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e0790340 con 0x55f5e03b40a0 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.386+0000 7f4d77400640 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e9c0 tx=0x7f4d6800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc00d99cc7a1aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d67400640 1 -- 172.21.15.27:0/138093501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680ad380 con 0x55f5e03b40a0 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d67400640 1 -- 172.21.15.27:0/138093501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d680b22a0 con 0x55f5e03b40a0 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d67400640 1 -- 172.21.15.27:0/138093501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680a91c0 con 0x55f5e03b40a0 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 -- 172.21.15.27:0/138093501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 msgr2=0x55f5e078f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e9c0 tx=0x7f4d6800ce60 comp rx=0 tx=0).stop 2024-09-21T01:22:02.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 -- 172.21.15.27:0/138093501 shutdown_connections 2024-09-21T01:22:02.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03dcf60 0x55f5e078a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e078f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 --2- 172.21.15.27:0/138093501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03b40a0 0x55f5e078f260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.387+0000 7f4d84e95740 1 -- 172.21.15.27:0/138093501 >> 172.21.15.27:0/138093501 conn(0x55f5e03d1350 msgr2=0x55f5e0780ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.388+0000 7f4d84e95740 1 -- 172.21.15.27:0/138093501 shutdown_connections 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.388+0000 7f4d84e95740 1 -- 172.21.15.27:0/138093501 wait complete. 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.388+0000 7f4d84e95740 1 -- start start 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.388+0000 7f4d84e95740 1 Processor -- start 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03b40a0 0x55f5e07a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e07a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e07a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f5e07a10d0 con 0x55f5e03dcf60 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f5e07a0a00 con 0x55f5e03bc280 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f5e07a17a0 con 0x55f5e03b40a0 2024-09-21T01:22:02.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d84e95740 10 client.0 _ll_register_callbacks cb 0x7f4d700099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e07a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d77400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03b40a0 0x55f5e07a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e07a39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37178/0 (socket says 172.21.15.27:37178) 2024-09-21T01:22:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e07a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.389+0000 7f4d77e00640 1 -- 172.21.15.27:0/310697397 learned_addr learned my addr 172.21.15.27:0/310697397 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:02.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d77e00640 1 -- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03b40a0 msgr2=0x55f5e07a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d77e00640 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03b40a0 0x55f5e07a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d77e00640 1 -- 172.21.15.27:0/310697397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 msgr2=0x55f5e07a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d77e00640 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e07a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d77e00640 1 -- 172.21.15.27:0/310697397 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e07a4130 con 0x55f5e03dcf60 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d6fe00640 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e07a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d77e00640 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e07a39a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4d70030e90 tx=0x7f4d70003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd11fe661c657062 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d64c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d64c00640 1 -- 172.21.15.27:0/310697397 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70003970 con 0x55f5e03dcf60 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d64c00640 1 -- 172.21.15.27:0/310697397 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4d700307d0 con 0x55f5e03dcf60 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.390+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d680b22a0 con 0x55f5e03dcf60 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d64c00640 1 -- 172.21.15.27:0/310697397 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d70078050 con 0x55f5e03dcf60 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 10 client.15351 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:02.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5e080c7e0 con 0x55f5e03dcf60 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d64c00640 1 -- 172.21.15.27:0/310697397 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d680b22a0 con 0x55f5e03dcf60 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d64c00640 1 -- 172.21.15.27:0/310697397 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4d70073030 con 0x55f5e03dcf60 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 10 client.15351 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 20 client.15351 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e0370c90 0x55f5e08100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 4 client.15351 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.391+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 --> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] -- command(tid 0: {"prefix": "status"}) -- 0x55f5e06f8750 con 0x55f5e0370c90 2024-09-21T01:22:02.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.392+0000 7f4d77400640 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e0370c90 0x55f5e08100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.392+0000 7f4d77400640 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e0370c90 0x55f5e08100b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7752b1194e890083 server_cookie=267ddfa889d68afd in_seq=0 out_seq=0 2024-09-21T01:22:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.392+0000 7f4d64c00640 10 client.15351 ms_handle_connect on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d64c00640 1 -- 172.21.15.27:0/310697397 <== mds.3 v2:172.21.15.98:6838/3799755243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f5e06f8750 con 0x55f5e0370c90 2024-09-21T01:22:02.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d64c00640 10 client.15351 handle_command_reply: tid=0 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 client.15351 shutdown 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 msgr2=0x55f5e07a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e07a39a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4d70030e90 tx=0x7f4d70003f80 comp rx=0 tx=0).stop 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 shutdown_connections 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55f5e0370c90 0x55f5e08100b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f5e03dcf60 0x55f5e07a39a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f5e03bc280 0x55f5e07a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 --2- 172.21.15.27:0/310697397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f5e03b40a0 0x55f5e07a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 >> 172.21.15.27:0/310697397 conn(0x55f5e03d1350 msgr2=0x55f5e078a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d64c00640 0 client.15351 ms_handle_reset on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 shutdown_connections 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 1 -- 172.21.15.27:0/310697397 wait complete. 2024-09-21T01:22:02.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.393+0000 7f4d84e95740 20 client.15351 trim_cache size 0 max 16384 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24730, 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 119.024815606, 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:02.398 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:22:02.399 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 199.06425285399999 2024-09-21T01:22:02.399 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4d5c0b65e0 msgr2=0x7f4d5c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4d5c0b65e0 0x7f4d5c0b8af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4d7002ade0 tx=0x7f4d7002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d78105bb0 msgr2=0x7f4d7819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d78105bb0 0x7f4d7819c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e9f0 tx=0x7f4d6800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 shutdown_connections 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4d5c0b65e0 0x7f4d5c0b8af0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4d7810be40 0x7f4d781a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7810b4f0 0x7f4d7819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 --2- 172.21.15.27:0/3215421491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d78105bb0 0x7f4d7819c5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.394+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 >> 172.21.15.27:0/3215421491 conn(0x7f4d78101680 msgr2=0x7f4d7810aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.395+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 shutdown_connections 2024-09-21T01:22:02.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:02.395+0000 7f4d7cc00640 1 -- 172.21.15.27:0/3215421491 wait complete. 2024-09-21T01:22:03.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T01:22:03.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:03 smithi027 ceph-mon[30375]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:03.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:03 smithi098 ceph-mon[34372]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:04.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:03 smithi089 ceph-mon[34486]: pgmap v389: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.536+0000 7ff58f200640 1 -- 172.21.15.27:0/3118701323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 msgr2=0x7ff58810f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.536+0000 7ff58f200640 1 --2- 172.21.15.27:0/3118701323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 0x7ff58810f600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff580009be0 tx=0x7ff58002a6c0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 -- 172.21.15.27:0/3118701323 shutdown_connections 2024-09-21T01:22:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 --2- 172.21.15.27:0/3118701323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 0x7ff58810f600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 --2- 172.21.15.27:0/3118701323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 0x7ff588108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 --2- 172.21.15.27:0/3118701323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 0x7ff588102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 -- 172.21.15.27:0/3118701323 >> 172.21.15.27:0/3118701323 conn(0x7ff588079750 msgr2=0x7ff588079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:04.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 -- 172.21.15.27:0/3118701323 shutdown_connections 2024-09-21T01:22:04.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.537+0000 7ff58f200640 1 -- 172.21.15.27:0/3118701323 wait complete. 2024-09-21T01:22:04.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.538+0000 7ff58f200640 1 Processor -- start 2024-09-21T01:22:04.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.538+0000 7ff58f200640 1 -- start start 2024-09-21T01:22:04.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58f200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 0x7ff58819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 0x7ff58819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58f200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 0x7ff58819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5881129a0 con 0x7ff588108480 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58f200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff588112820 con 0x7ff588102b40 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58f200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff588112b20 con 0x7ff588108dd0 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff57f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 0x7ff58819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff57f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 0x7ff58819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37204/0 (socket says 172.21.15.27:37204) 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff57f400640 1 -- 172.21.15.27:0/3712302208 learned_addr learned my addr 172.21.15.27:0/3712302208 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff587e00640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 0x7ff58819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.539+0000 7ff58d200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 0x7ff58819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff587e00640 1 -- 172.21.15.27:0/3712302208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 msgr2=0x7ff58819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff587e00640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 0x7ff58819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff587e00640 1 -- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 msgr2=0x7ff58819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff587e00640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 0x7ff58819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff587e00640 1 -- 172.21.15.27:0/3712302208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5881a0350 con 0x7ff588102b40 2024-09-21T01:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff587e00640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 0x7ff58819a3e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff57800e9f0 tx=0x7ff57800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff585600640 1 -- 172.21.15.27:0/3712302208 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800ccc0 con 0x7ff588102b40 2024-09-21T01:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff585600640 1 -- 172.21.15.27:0/3712302208 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff578005020 con 0x7ff588102b40 2024-09-21T01:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5881a0640 con 0x7ff588102b40 2024-09-21T01:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.540+0000 7ff585600640 1 -- 172.21.15.27:0/3712302208 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800ce60 con 0x7ff588102b40 2024-09-21T01:22:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.541+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5881a6da0 con 0x7ff588102b40 2024-09-21T01:22:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 Processor -- start 2024-09-21T01:22:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 -- start start 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f90927f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb00a0 0x55f90927f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f90927ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f9091f6d50 con 0x55f908eb8280 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f9091e6180 con 0x55f908eb00a0 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff595e83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f9091e9f90 con 0x55f908ed8f60 2024-09-21T01:22:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.544+0000 7ff585600640 1 -- 172.21.15.27:0/3712302208 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7ff5780051d0 con 0x7ff588102b40 2024-09-21T01:22:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff57f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f90927ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f90927f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f90927f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:53280/0 (socket says 172.21.15.27:53280) 2024-09-21T01:22:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 -- 172.21.15.27:0/1062680117 learned_addr learned my addr 172.21.15.27:0/1062680117 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff57f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f90927ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37208/0 (socket says 172.21.15.27:37208) 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff587e00640 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb00a0 0x55f90927f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 -- 172.21.15.27:0/1062680117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb00a0 msgr2=0x55f90927f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb00a0 0x55f90927f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 -- 172.21.15.27:0/1062680117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 msgr2=0x55f90927ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f90927ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.545+0000 7ff58d200640 1 -- 172.21.15.27:0/1062680117 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9091b0a70 con 0x55f908ed8f60 2024-09-21T01:22:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.546+0000 7ff58d200640 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f90927f490 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff580009be0 tx=0x7ff580030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef8de58edbb97fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.546+0000 7ff57d600640 1 -- 172.21.15.27:0/1062680117 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58004a070 con 0x55f908ed8f60 2024-09-21T01:22:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.546+0000 7ff57d600640 1 -- 172.21.15.27:0/1062680117 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff580003e20 con 0x55f908ed8f60 2024-09-21T01:22:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.546+0000 7ff57d600640 1 -- 172.21.15.27:0/1062680117 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580036040 con 0x55f908ed8f60 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff585600640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff56c0b6560 0x7ff56c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff585600640 1 -- 172.21.15.27:0/3712302208 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff5780e7740 con 0x7ff588102b40 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff587e00640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff56c0b6560 0x7ff56c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 -- 172.21.15.27:0/1062680117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 msgr2=0x55f90927f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f90927f490 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff580009be0 tx=0x7ff580030430 comp rx=0 tx=0).stop 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 -- 172.21.15.27:0/1062680117 shutdown_connections 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f90927ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb00a0 0x55f90927f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 --2- 172.21.15.27:0/1062680117 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f90927f490 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.547+0000 7ff595e83740 1 -- 172.21.15.27:0/1062680117 >> 172.21.15.27:0/1062680117 conn(0x55f908ecd350 msgr2=0x55f90927d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.548+0000 7ff595e83740 1 -- 172.21.15.27:0/1062680117 shutdown_connections 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.548+0000 7ff595e83740 1 -- 172.21.15.27:0/1062680117 wait complete. 2024-09-21T01:22:04.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.548+0000 7ff587e00640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff56c0b6560 0x7ff56c0b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5780aa6b0 tx=0x7ff5780ed550 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.548+0000 7ff595e83740 1 -- start start 2024-09-21T01:22:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.548+0000 7ff595e83740 1 Processor -- start 2024-09-21T01:22:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90928b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 0x55f90928b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f909296f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f9090475f0 con 0x55f908eb00a0 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f909181460 con 0x55f908eb8280 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f90928dc10 con 0x55f908ed8f60 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff595e83740 10 client.0 _ll_register_callbacks cb 0x55f909191210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff58d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90928b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff57f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 0x55f90928b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff58d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90928b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37224/0 (socket says 172.21.15.27:37224) 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff57f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 0x55f90928b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35738/0 (socket says 172.21.15.27:35738) 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.549+0000 7ff58d200640 1 -- 172.21.15.27:0/4094829031 learned_addr learned my addr 172.21.15.27:0/4094829031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff58d200640 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f909296f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff57f400640 1 -- 172.21.15.27:0/4094829031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 msgr2=0x55f909296f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff57f400640 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f909296f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff57f400640 1 -- 172.21.15.27:0/4094829031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 msgr2=0x55f90928b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff57f400640 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90928b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff57f400640 1 -- 172.21.15.27:0/4094829031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f908f44540 con 0x55f908eb8280 2024-09-21T01:22:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff57f400640 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 0x55f90928b550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff570009fd0 tx=0x7ff570012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3424422fa09044e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.550+0000 7ff576000640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff576000640 1 -- 172.21.15.27:0/4094829031 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570004030 con 0x55f908eb8280 2024-09-21T01:22:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff576000640 1 -- 172.21.15.27:0/4094829031 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff5700041d0 con 0x55f908eb8280 2024-09-21T01:22:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f909183a70 con 0x55f908eb8280 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff576000640 1 -- 172.21.15.27:0/4094829031 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff570004410 con 0x55f908eb8280 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff595e83740 10 client.25403 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f90919e6f0 con 0x55f908eb8280 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff576000640 1 -- 172.21.15.27:0/4094829031 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f909183a70 con 0x55f908eb8280 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.551+0000 7ff576000640 1 -- 172.21.15.27:0/4094829031 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff570003b80 con 0x55f908eb8280 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff595e83740 10 client.25403 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff595e83740 10 client.25403 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff595e83740 20 client.25403 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff595e83740 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e978e0 0x55f90929d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff595e83740 4 client.25403 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 --> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9091f8ca0 con 0x55f908e978e0 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.552+0000 7ff58d200640 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e978e0 0x55f90929d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.553+0000 7ff58d200640 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e978e0 0x55f90929d540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0fe464f477ecad1 server_cookie=271ff7232ab5048c in_seq=0 out_seq=0 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.553+0000 7ff576000640 10 client.25403 ms_handle_connect on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.553+0000 7ff576000640 1 -- 172.21.15.27:0/4094829031 <== mds.2 v2:172.21.15.89:6836/3556269004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9091f8ca0 con 0x55f908e978e0 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.553+0000 7ff576000640 10 client.25403 handle_command_reply: tid=0 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 client.25403 shutdown 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 msgr2=0x55f90928b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 0x55f90928b550 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff570009fd0 tx=0x7ff570012bd0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 shutdown_connections 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e978e0 0x55f90929d540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908ed8f60 0x55f909296f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908eb8280 0x55f90928b550 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 --2- 172.21.15.27:0/4094829031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90928b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 >> 172.21.15.27:0/4094829031 conn(0x55f908ecd350 msgr2=0x55f909282620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff576000640 0 client.25403 ms_handle_reset on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 shutdown_connections 2024-09-21T01:22:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 1 -- 172.21.15.27:0/4094829031 wait complete. 2024-09-21T01:22:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.554+0000 7ff595e83740 20 client.25403 trim_cache size 0 max 16384 2024-09-21T01:22:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 Processor -- start 2024-09-21T01:22:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 -- start start 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90929c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb8280 0x55f909296e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f909285f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f90929b880 con 0x55f908eb00a0 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f90929bf50 con 0x55f908ed8f60 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.573+0000 7ff595e83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f90929c620 con 0x55f908eb8280 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90929c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90929c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37226/0 (socket says 172.21.15.27:37226) 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 -- 172.21.15.27:0/2604136491 learned_addr learned my addr 172.21.15.27:0/2604136491 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff57f400640 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb8280 0x55f909296e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f909285f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 -- 172.21.15.27:0/2604136491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb8280 msgr2=0x55f909296e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb8280 0x55f909296e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 -- 172.21.15.27:0/2604136491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 msgr2=0x55f909285f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f909285f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.574+0000 7ff58d200640 1 -- 172.21.15.27:0/2604136491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f909281780 con 0x55f908eb00a0 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.575+0000 7ff58d200640 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90929c0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff580030cd0 tx=0x7ff580030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2865d6514a1c1326 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.575+0000 7ff57cc00640 1 -- 172.21.15.27:0/2604136491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580036070 con 0x55f908eb00a0 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.575+0000 7ff57cc00640 1 -- 172.21.15.27:0/2604136491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff580030770 con 0x55f908eb00a0 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.575+0000 7ff57cc00640 1 -- 172.21.15.27:0/2604136491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58004a040 con 0x55f908eb00a0 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 -- 172.21.15.27:0/2604136491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 msgr2=0x55f90929c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90929c0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff580030cd0 tx=0x7ff580030c60 comp rx=0 tx=0).stop 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 -- 172.21.15.27:0/2604136491 shutdown_connections 2024-09-21T01:22:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f909285f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb8280 0x55f909296e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 --2- 172.21.15.27:0/2604136491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb00a0 0x55f90929c0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 -- 172.21.15.27:0/2604136491 >> 172.21.15.27:0/2604136491 conn(0x55f908ecd350 msgr2=0x55f909278010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 -- 172.21.15.27:0/2604136491 shutdown_connections 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 -- 172.21.15.27:0/2604136491 wait complete. 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.576+0000 7ff595e83740 1 -- start start 2024-09-21T01:22:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 Processor -- start 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb00a0 0x55f90929ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f9092a0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f9092a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f90929d3c0 con 0x55f908eb8280 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f90929ccf0 con 0x55f908ed8f60 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f90929da90 con 0x55f908eb00a0 2024-09-21T01:22:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff595e83740 10 client.0 _ll_register_callbacks cb 0x7ff570009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff57f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f9092a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff57f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f9092a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37240/0 (socket says 172.21.15.27:37240) 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.577+0000 7ff57f400640 1 -- 172.21.15.27:0/2742456166 learned_addr learned my addr 172.21.15.27:0/2742456166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff58d200640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb00a0 0x55f90929ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff58d200640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f9092a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff57f400640 1 -- 172.21.15.27:0/2742456166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb00a0 msgr2=0x55f90929ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff57f400640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb00a0 0x55f90929ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff57f400640 1 -- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 msgr2=0x55f9092a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff57f400640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f9092a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff57f400640 1 -- 172.21.15.27:0/2742456166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9092a52b0 con 0x55f908eb8280 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff58d200640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f9092a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:22:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff57f400640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f9092a0460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff570001eb0 tx=0x7ff5700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c44e1106b16ec1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.578+0000 7ff576000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff576000640 1 -- 172.21.15.27:0/2742456166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000fb30 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff580030770 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff576000640 1 -- 172.21.15.27:0/2742456166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff57000fcd0 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff576000640 1 -- 172.21.15.27:0/2742456166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5700035f0 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff595e83740 10 client.15369 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9092a6590 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff576000640 1 -- 172.21.15.27:0/2742456166 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff580030770 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff576000640 1 -- 172.21.15.27:0/2742456166 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff5700108c0 con 0x55f908eb8280 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff595e83740 10 client.15369 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.579+0000 7ff595e83740 10 client.15369 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff595e83740 20 client.15369 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff595e83740 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e6cc90 0x55f90930c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff595e83740 4 client.15369 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 --> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] -- command(tid 0: {"prefix": "status"}) -- 0x55f9091f4750 con 0x55f908e6cc90 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff58d200640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e6cc90 0x55f90930c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff58d200640 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e6cc90 0x55f90930c130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ced5b4be2546dd92 server_cookie=ec70cb6090b0f00f in_seq=0 out_seq=0 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.580+0000 7ff576000640 10 client.15369 ms_handle_connect on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff576000640 1 -- 172.21.15.27:0/2742456166 <== mds.2 v2:172.21.15.89:6836/3556269004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f9091f4750 con 0x55f908e6cc90 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff576000640 10 client.15369 handle_command_reply: tid=0 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 client.15369 shutdown 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 msgr2=0x55f9092a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f9092a0460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff570001eb0 tx=0x7ff5700057f0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 shutdown_connections 2024-09-21T01:22:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x55f908e6cc90 0x55f90930c130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f908ed8f60 0x55f9092a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f908eb8280 0x55f9092a0460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 --2- 172.21.15.27:0/2742456166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f908eb00a0 0x55f90929ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 >> 172.21.15.27:0/2742456166 conn(0x55f908ecd350 msgr2=0x55f909285f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff576000640 0 client.15369 ms_handle_reset on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 shutdown_connections 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 1 -- 172.21.15.27:0/2742456166 wait complete. 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.581+0000 7ff595e83740 20 client.15369 trim_cache size 0 max 16384 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:04.586 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24734, 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 122.224163194, 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 203.90048347499999 2024-09-21T01:22:04.587 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff56c0b6560 msgr2=0x7ff56c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff56c0b6560 0x7ff56c0b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5780aa6b0 tx=0x7ff5780ed550 comp rx=0 tx=0).stop 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 msgr2=0x7ff58819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 0x7ff58819a3e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff57800e9f0 tx=0x7ff57800edf0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 shutdown_connections 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff56c0b6560 0x7ff56c0b8a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff588108dd0 0x7ff58819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff588108480 0x7ff58819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 --2- 172.21.15.27:0/3712302208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff588102b40 0x7ff58819a3e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 >> 172.21.15.27:0/3712302208 conn(0x7ff588079750 msgr2=0x7ff58806be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 shutdown_connections 2024-09-21T01:22:04.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:04.582+0000 7ff58f200640 1 -- 172.21.15.27:0/3712302208 wait complete. 2024-09-21T01:22:05.644 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.g status 2024-09-21T01:22:05.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:05 smithi098 ceph-mon[34372]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:22:05.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:05 smithi027 ceph-mon[30375]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:22:06.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:05 smithi089 ceph-mon[34486]: pgmap v390: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:22:06.029 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 -- 172.21.15.27:0/1066893443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd40109c40 msgr2=0x7efd40110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 --2- 172.21.15.27:0/1066893443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd40109c40 0x7efd40110470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efd38009ca0 tx=0x7efd3802a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 -- 172.21.15.27:0/1066893443 shutdown_connections 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 --2- 172.21.15.27:0/1066893443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd40109c40 0x7efd40110470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 --2- 172.21.15.27:0/1066893443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd401092f0 0x7efd40109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 --2- 172.21.15.27:0/1066893443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd401039b0 0x7efd40103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 -- 172.21.15.27:0/1066893443 >> 172.21.15.27:0/1066893443 conn(0x7efd400ff400 msgr2=0x7efd40101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 -- 172.21.15.27:0/1066893443 shutdown_connections 2024-09-21T01:22:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.841+0000 7efd44e00640 1 -- 172.21.15.27:0/1066893443 wait complete. 2024-09-21T01:22:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.842+0000 7efd44e00640 1 Processor -- start 2024-09-21T01:22:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.842+0000 7efd44e00640 1 -- start start 2024-09-21T01:22:06.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd44e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd401039b0 0x7efd40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd44e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 0x7efd40075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd44e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd40109c40 0x7efd40076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd44e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efd40113900 con 0x7efd401092f0 2024-09-21T01:22:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd44e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd40113780 con 0x7efd401039b0 2024-09-21T01:22:06.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd44e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd40113a80 con 0x7efd40109c40 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 0x7efd40075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 0x7efd40075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32898/0 (socket says 172.21.15.27:32898) 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd3ea00640 1 -- 172.21.15.27:0/2603075290 learned_addr learned my addr 172.21.15.27:0/2603075290 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd3f400640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd401039b0 0x7efd40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.843+0000 7efd3fe00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd40109c40 0x7efd40076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd3ea00640 1 -- 172.21.15.27:0/2603075290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd40109c40 msgr2=0x7efd40076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd3ea00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd40109c40 0x7efd40076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd3ea00640 1 -- 172.21.15.27:0/2603075290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd401039b0 msgr2=0x7efd40075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd3ea00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd401039b0 0x7efd40075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd3ea00640 1 -- 172.21.15.27:0/2603075290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd401a84f0 con 0x7efd401092f0 2024-09-21T01:22:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd3ea00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 0x7efd40075c50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00e970 tx=0x7efd2c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd37e00640 1 -- 172.21.15.27:0/2603075290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c00cd80 con 0x7efd401092f0 2024-09-21T01:22:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd37e00640 1 -- 172.21.15.27:0/2603075290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c0035f0 con 0x7efd401092f0 2024-09-21T01:22:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd37e00640 1 -- 172.21.15.27:0/2603075290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c010690 con 0x7efd401092f0 2024-09-21T01:22:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.844+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd401a87e0 con 0x7efd401092f0 2024-09-21T01:22:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.845+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401a8d70 con 0x7efd401092f0 2024-09-21T01:22:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.849+0000 7efd4d0a0740 1 Processor -- start 2024-09-21T01:22:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.849+0000 7efd4d0a0740 1 -- start start 2024-09-21T01:22:06.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.849+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebe5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.849+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebe5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebe60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a1eb5cd50 con 0x561a1e8160a0 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561a1eb4c180 con 0x561a1e83ef60 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561a1eb4ff90 con 0x561a1e81e280 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd37e00640 1 -- 172.21.15.27:0/2603075290 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7efd2c003aa0 con 0x7efd401092f0 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebe5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebe5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45650/0 (socket says 172.21.15.27:45650) 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 -- 172.21.15.27:0/156513780 learned_addr learned my addr 172.21.15.27:0/156513780 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3f400640 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebe5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3ea00640 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebe60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 -- 172.21.15.27:0/156513780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 msgr2=0x561a1ebe60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebe60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 -- 172.21.15.27:0/156513780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 msgr2=0x561a1ebe5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebe5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.850+0000 7efd3fe00640 1 -- 172.21.15.27:0/156513780 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1eb16a70 con 0x561a1e83ef60 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.851+0000 7efd3fe00640 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebe5670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd3802f410 tx=0x7efd3802f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1c6eb578e240c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.851+0000 7efd34c00640 1 -- 172.21.15.27:0/156513780 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38031cf0 con 0x561a1e83ef60 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.851+0000 7efd34c00640 1 -- 172.21.15.27:0/156513780 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd38036030 con 0x561a1e83ef60 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.851+0000 7efd34c00640 1 -- 172.21.15.27:0/156513780 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380042a0 con 0x561a1e83ef60 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd37e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd240b65e0 0x7efd240b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 -- 172.21.15.27:0/156513780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 msgr2=0x561a1ebe5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebe5670 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efd3802f410 tx=0x7efd3802f440 comp rx=0 tx=0).stop 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd37e00640 1 -- 172.21.15.27:0/2603075290 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efd2c0e7a50 con 0x7efd401092f0 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 -- 172.21.15.27:0/156513780 shutdown_connections 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebe60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebe5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/156513780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebe5670 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd4d0a0740 1 -- 172.21.15.27:0/156513780 >> 172.21.15.27:0/156513780 conn(0x561a1e833350 msgr2=0x561a1ebe3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.852+0000 7efd3f400640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd240b65e0 0x7efd240b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.853+0000 7efd4d0a0740 1 -- 172.21.15.27:0/156513780 shutdown_connections 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.853+0000 7efd4d0a0740 1 -- 172.21.15.27:0/156513780 wait complete. 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.853+0000 7efd3f400640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd240b65e0 0x7efd240b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x561a1ebe42f0 tx=0x7efd30005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.853+0000 7efd4d0a0740 1 -- start start 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.853+0000 7efd4d0a0740 1 Processor -- start 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebf12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebf17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebfcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a1e9ad5f0 con 0x561a1e8160a0 2024-09-21T01:22:06.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561a1eae7460 con 0x561a1e83ef60 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561a1ebf3c70 con 0x561a1e81e280 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd4d0a0740 10 client.0 _ll_register_callbacks cb 0x561a1eaf7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd3f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebf12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd3f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebf12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32920/0 (socket says 172.21.15.27:32920) 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.854+0000 7efd3f400640 1 -- 172.21.15.27:0/1403706775 learned_addr learned my addr 172.21.15.27:0/1403706775 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3fe00640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebfcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebf17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3f400640 1 -- 172.21.15.27:0/1403706775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 msgr2=0x561a1ebf17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3f400640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebf17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3f400640 1 -- 172.21.15.27:0/1403706775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 msgr2=0x561a1ebfcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3f400640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebfcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3f400640 1 -- 172.21.15.27:0/1403706775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1e8aa540 con 0x561a1e8160a0 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3fe00640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebfcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:06.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd3f400640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebf12a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efd30004620 tx=0x7efd30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75242185d99b0323 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:06.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.855+0000 7efd29600640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd29600640 1 -- 172.21.15.27:0/1403706775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30009e20 con 0x561a1e8160a0 2024-09-21T01:22:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd29600640 1 -- 172.21.15.27:0/1403706775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efd30005020 con 0x561a1e8160a0 2024-09-21T01:22:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a1eae9a70 con 0x561a1e8160a0 2024-09-21T01:22:06.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd29600640 1 -- 172.21.15.27:0/1403706775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd30005220 con 0x561a1e8160a0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd4d0a0740 10 client.15381 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a1eb046f0 con 0x561a1e8160a0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.856+0000 7efd29600640 1 -- 172.21.15.27:0/1403706775 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a1eae9a70 con 0x561a1e8160a0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd29600640 1 -- 172.21.15.27:0/1403706775 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7efd30027070 con 0x561a1e8160a0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd4d0a0740 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd4d0a0740 10 client.15381 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd4d0a0740 20 client.15381 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7fd8e0 0x561a1ec036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd4d0a0740 4 client.15381 mds_command: new command op to 24752 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 --> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a1eb5eca0 con 0x561a1e7fd8e0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.857+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7fd8e0 0x561a1ec036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.858+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7fd8e0 0x561a1ec036a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c85a752134191c8a server_cookie=fe77e14e66f62359 in_seq=0 out_seq=0 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.858+0000 7efd29600640 10 client.15381 ms_handle_connect on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:06.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd29600640 1 -- 172.21.15.27:0/1403706775 <== mds.1 v2:172.21.15.27:6838/3115497172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561a1eb5eca0 con 0x561a1e7fd8e0 2024-09-21T01:22:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd29600640 10 client.15381 handle_command_reply: tid=0 2024-09-21T01:22:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 client.15381 shutdown 2024-09-21T01:22:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 msgr2=0x561a1ebf12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebf12a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efd30004620 tx=0x7efd30004730 comp rx=0 tx=0).stop 2024-09-21T01:22:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 shutdown_connections 2024-09-21T01:22:06.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7fd8e0 0x561a1ec036a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e83ef60 0x561a1ebfcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ebf17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1403706775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e8160a0 0x561a1ebf12a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd29600640 0 client.15381 ms_handle_reset on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 >> 172.21.15.27:0/1403706775 conn(0x561a1e833350 msgr2=0x561a1ebe8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 shutdown_connections 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1403706775 wait complete. 2024-09-21T01:22:06.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.859+0000 7efd4d0a0740 20 client.15381 trim_cache size 0 max 16384 2024-09-21T01:22:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.878+0000 7efd4d0a0740 1 Processor -- start 2024-09-21T01:22:06.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 -- start start 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec01b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ec036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e83ef60 0x561a1ec01060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a1ec019e0 con 0x561a1e83ef60 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561a1ec020b0 con 0x561a1e8160a0 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561a1ec02780 con 0x561a1e81e280 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e83ef60 0x561a1ec01060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec01b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e83ef60 0x561a1ec01060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32936/0 (socket says 172.21.15.27:32936) 2024-09-21T01:22:06.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd3ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ec036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.879+0000 7efd3fe00640 1 -- 172.21.15.27:0/1030825160 learned_addr learned my addr 172.21.15.27:0/1030825160 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd3ea00640 1 -- 172.21.15.27:0/1030825160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 msgr2=0x561a1ec01b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec01b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd3ea00640 1 -- 172.21.15.27:0/1030825160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e83ef60 msgr2=0x561a1ec01060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e83ef60 0x561a1ec01060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd3ea00640 1 -- 172.21.15.27:0/1030825160 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1ebe7960 con 0x561a1e81e280 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ec036a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0058c0 tx=0x7efd2c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=613065fe5c30215b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:06.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd2be00640 1 -- 172.21.15.27:0/1030825160 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0ad200 con 0x561a1e81e280 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd2be00640 1 -- 172.21.15.27:0/1030825160 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c0addf0 con 0x561a1e81e280 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.880+0000 7efd2be00640 1 -- 172.21.15.27:0/1030825160 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0a92a0 con 0x561a1e81e280 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1030825160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 msgr2=0x561a1ec036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ec036a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0058c0 tx=0x7efd2c000bc0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1030825160 shutdown_connections 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e83ef60 0x561a1ec01060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e81e280 0x561a1ec036a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1030825160 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec01b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1030825160 >> 172.21.15.27:0/1030825160 conn(0x561a1e833350 msgr2=0x561a1ebde2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1030825160 shutdown_connections 2024-09-21T01:22:06.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.881+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1030825160 wait complete. 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.882+0000 7efd4d0a0740 1 -- start start 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.882+0000 7efd4d0a0740 1 Processor -- start 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec0bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e81e280 0x561a1ec0c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e83ef60 0x561a1ec05220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a1ec03520 con 0x561a1e81e280 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561a1ec02e50 con 0x561a1e8160a0 2024-09-21T01:22:06.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561a1ec03bf0 con 0x561a1e83ef60 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd4d0a0740 10 client.0 _ll_register_callbacks cb 0x7efd300044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec0bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec0bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45672/0 (socket says 172.21.15.27:45672) 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 -- 172.21.15.27:0/1845352935 learned_addr learned my addr 172.21.15.27:0/1845352935 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e81e280 0x561a1ec0c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3fe00640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e83ef60 0x561a1ec05220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 -- 172.21.15.27:0/1845352935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e83ef60 msgr2=0x561a1ec05220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e83ef60 0x561a1ec05220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 -- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e81e280 msgr2=0x561a1ec0c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e81e280 0x561a1ec0c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.883+0000 7efd3f400640 1 -- 172.21.15.27:0/1845352935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1ec059b0 con 0x561a1e8160a0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd3f400640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec0bcb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efd300092e0 tx=0x7efd30005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08477b6cf9108bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd29600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:06.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd29600640 1 -- 172.21.15.27:0/1845352935 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300047a0 con 0x561a1e8160a0 2024-09-21T01:22:06.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd29600640 1 -- 172.21.15.27:0/1845352935 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efd30004940 con 0x561a1e8160a0 2024-09-21T01:22:06.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd2c0addf0 con 0x561a1e8160a0 2024-09-21T01:22:06.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd29600640 1 -- 172.21.15.27:0/1845352935 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd30004b40 con 0x561a1e8160a0 2024-09-21T01:22:06.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd4d0a0740 10 client.25427 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:06.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.884+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a1ec0b440 con 0x561a1e8160a0 2024-09-21T01:22:06.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd29600640 1 -- 172.21.15.27:0/1845352935 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c0addf0 con 0x561a1e8160a0 2024-09-21T01:22:06.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd29600640 1 -- 172.21.15.27:0/1845352935 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7efd30027040 con 0x561a1e8160a0 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd4d0a0740 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd4d0a0740 10 client.25427 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd4d0a0740 20 client.25427 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7d2c90 0x561a1ec71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd4d0a0740 4 client.25427 mds_command: new command op to 24752 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 --> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] -- command(tid 0: {"prefix": "status"}) -- 0x561a1eb5a750 con 0x561a1e7d2c90 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.885+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7d2c90 0x561a1ec71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd3ea00640 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7d2c90 0x561a1ec71fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bf6e8140e2308c5 server_cookie=e399ff0c980adab6 in_seq=0 out_seq=0 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd29600640 10 client.25427 ms_handle_connect on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd29600640 1 -- 172.21.15.27:0/1845352935 <== mds.1 v2:172.21.15.27:6838/3115497172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x561a1eb5a750 con 0x561a1e7d2c90 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd29600640 10 client.25427 handle_command_reply: tid=0 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 client.25427 shutdown 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 msgr2=0x561a1ec0bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec0bcb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efd300092e0 tx=0x7efd30005b20 comp rx=0 tx=0).stop 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 shutdown_connections 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x561a1e7d2c90 0x561a1ec71fe0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561a1e83ef60 0x561a1ec05220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd29600640 0 client.25427 ms_handle_reset on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.886+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a1e81e280 0x561a1ec0c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.887+0000 7efd4d0a0740 1 --2- 172.21.15.27:0/1845352935 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561a1e8160a0 0x561a1ec0bcb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.887+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 >> 172.21.15.27:0/1845352935 conn(0x561a1e833350 msgr2=0x561a1ec01060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.887+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 shutdown_connections 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.887+0000 7efd4d0a0740 1 -- 172.21.15.27:0/1845352935 wait complete. 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.887+0000 7efd4d0a0740 20 client.25427 trim_cache size 0 max 16384 2024-09-21T01:22:06.890 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24752, 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 125.538292239, 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 199.473649682 2024-09-21T01:22:06.891 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.887+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd240b65e0 msgr2=0x7efd240b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd240b65e0 0x7efd240b8af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x561a1ebe42f0 tx=0x7efd30005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 msgr2=0x7efd40075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 0x7efd40075c50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00e970 tx=0x7efd2c00ee30 comp rx=0 tx=0).stop 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 shutdown_connections 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efd240b65e0 0x7efd240b8af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd40109c40 0x7efd40076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd401092f0 0x7efd40075c50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 --2- 172.21.15.27:0/2603075290 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd401039b0 0x7efd40075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 >> 172.21.15.27:0/2603075290 conn(0x7efd400ff400 msgr2=0x7efd401068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:06.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 shutdown_connections 2024-09-21T01:22:06.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:06.888+0000 7efd44e00640 1 -- 172.21.15.27:0/2603075290 wait complete. 2024-09-21T01:22:07.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T01:22:07.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:07 smithi098 ceph-mon[34372]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:22:07.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:07 smithi027 ceph-mon[30375]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:22:07.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:07 smithi089 ceph-mon[34486]: pgmap v391: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:22:08.170 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 -- 172.21.15.27:0/1225379074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f3810be60 msgr2=0x7f3f38112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1225379074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f3810be60 0x7f3f38112690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009be0 tx=0x7f3f3002a6c0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 -- 172.21.15.27:0/1225379074 shutdown_connections 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1225379074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f3810be60 0x7f3f38112690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1225379074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f3810b510 0x7f3f3810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1225379074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f38105bd0 0x7f3f38105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 -- 172.21.15.27:0/1225379074 >> 172.21.15.27:0/1225379074 conn(0x7f3f38101620 msgr2=0x7f3f38103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 -- 172.21.15.27:0/1225379074 shutdown_connections 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.832+0000 7f3f3e800640 1 -- 172.21.15.27:0/1225379074 wait complete. 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 Processor -- start 2024-09-21T01:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 -- start start 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f38105bd0 0x7f3f3819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b510 0x7f3f3819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f3810be60 0x7f3f381a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38115b20 con 0x7f3f3810b510 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3f381159a0 con 0x7f3f3810be60 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.834+0000 7f3f3e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f38115ca0 con 0x7f3f38105bd0 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b510 0x7f3f3819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f38105bd0 0x7f3f3819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f38105bd0 0x7f3f3819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45874/0 (socket says 172.21.15.27:45874) 2024-09-21T01:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3d400640 1 -- 172.21.15.27:0/1094380911 learned_addr learned my addr 172.21.15.27:0/1094380911 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3de00640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f3810be60 0x7f3f381a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1094380911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f38105bd0 msgr2=0x7f3f3819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f38105bd0 0x7f3f3819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1094380911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f3810be60 msgr2=0x7f3f381a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f3810be60 0x7f3f381a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.835+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1094380911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f381a2400 con 0x7f3f3810b510 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.836+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b510 0x7f3f3819ca30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3f3806c860 tx=0x7f3f2400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.836+0000 7f3f36000640 1 -- 172.21.15.27:0/1094380911 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400cc40 con 0x7f3f3810b510 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.836+0000 7f3f36000640 1 -- 172.21.15.27:0/1094380911 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2400cde0 con 0x7f3f3810b510 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.836+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f381a26f0 con 0x7f3f3810b510 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.836+0000 7f3f36000640 1 -- 172.21.15.27:0/1094380911 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24018780 con 0x7f3f3810b510 2024-09-21T01:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.836+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f381a8e50 con 0x7f3f3810b510 2024-09-21T01:22:08.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 Processor -- start 2024-09-21T01:22:08.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 -- start start 2024-09-21T01:22:08.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f36000640 1 -- 172.21.15.27:0/1094380911 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f3f24010a90 con 0x7f3f3810b510 2024-09-21T01:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f248b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f248bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f248c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5576f2402d50 con 0x5576f20c4280 2024-09-21T01:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5576f23f2180 con 0x5576f20e4f60 2024-09-21T01:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.840+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5576f23f5f90 con 0x5576f20bc0a0 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f248b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f248b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45706/0 (socket says 172.21.15.27:45706) 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 -- 172.21.15.27:0/1301503532 learned_addr learned my addr 172.21.15.27:0/1301503532 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f248c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3d400640 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f248bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 -- 172.21.15.27:0/1301503532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 msgr2=0x5576f248bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f248bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 -- 172.21.15.27:0/1301503532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 msgr2=0x5576f248c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f248c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.841+0000 7f3f3de00640 1 -- 172.21.15.27:0/1301503532 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f23bca70 con 0x5576f20e4f60 2024-09-21T01:22:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.842+0000 7f3f3de00640 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f248b670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009be0 tx=0x7f3f30030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a675329816da9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.842+0000 7f3f2a600640 1 -- 172.21.15.27:0/1301503532 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30041030 con 0x5576f20e4f60 2024-09-21T01:22:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.842+0000 7f3f2a600640 1 -- 172.21.15.27:0/1301503532 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f30003e20 con 0x5576f20e4f60 2024-09-21T01:22:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.842+0000 7f3f2a600640 1 -- 172.21.15.27:0/1301503532 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30036040 con 0x5576f20e4f60 2024-09-21T01:22:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.842+0000 7f3f36000640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f200b6560 0x7f3f200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.842+0000 7f3f36000640 1 -- 172.21.15.27:0/1094380911 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3f240e7c40 con 0x7f3f3810b510 2024-09-21T01:22:08.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f3d400640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f200b6560 0x7f3f200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1301503532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 msgr2=0x5576f248b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f248b670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009be0 tx=0x7f3f30030430 comp rx=0 tx=0).stop 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1301503532 shutdown_connections 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f248c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f248bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1301503532 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f248b670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1301503532 >> 172.21.15.27:0/1301503532 conn(0x5576f20d9350 msgr2=0x5576f2489560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1301503532 shutdown_connections 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.843+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1301503532 wait complete. 2024-09-21T01:22:08.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.844+0000 7f3f3d400640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f200b6560 0x7f3f200b8a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5576f248a2f0 tx=0x7f3f2c009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.844+0000 7f3f46bf3740 1 -- start start 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.844+0000 7f3f46bf3740 1 Processor -- start 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f2497720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20e4f60 0x5576f24a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5576f22535f0 con 0x5576f20c4280 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5576f238d460 con 0x5576f20bc0a0 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5576f2499df0 con 0x5576f20e4f60 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f46bf3740 10 client.0 _ll_register_callbacks cb 0x5576f239d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f2497720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f3de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20e4f60 0x5576f24a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f3de00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20e4f60 0x5576f24a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45880/0 (socket says 172.21.15.27:45880) 2024-09-21T01:22:08.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f2497720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32976/0 (socket says 172.21.15.27:32976) 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f3de00640 1 -- 172.21.15.27:0/742162808 learned_addr learned my addr 172.21.15.27:0/742162808 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.845+0000 7f3f3d400640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3ca00640 1 -- 172.21.15.27:0/742162808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20e4f60 msgr2=0x5576f24a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20e4f60 0x5576f24a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3ca00640 1 -- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 msgr2=0x5576f24971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3ca00640 1 -- 172.21.15.27:0/742162808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f2150540 con 0x5576f20c4280 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3d400640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f2497720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c6a0 tx=0x7f3f240acf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f9c748b92e0785 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:08.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f1ea00640 1 -- 172.21.15.27:0/742162808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240ad050 con 0x5576f20c4280 2024-09-21T01:22:08.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.846+0000 7f3f1ea00640 1 -- 172.21.15.27:0/742162808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3f240b3550 con 0x5576f20c4280 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576f238fa70 con 0x5576f20c4280 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f1ea00640 1 -- 172.21.15.27:0/742162808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f240ad3f0 con 0x5576f20c4280 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f46bf3740 10 client.15399 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576f23aa6f0 con 0x5576f20c4280 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f1ea00640 1 -- 172.21.15.27:0/742162808 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5576f238fa70 con 0x5576f20c4280 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f1ea00640 1 -- 172.21.15.27:0/742162808 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3f240b3750 con 0x5576f20c4280 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.847+0000 7f3f46bf3740 10 client.15399 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.848+0000 7f3f46bf3740 10 client.15399 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.848+0000 7f3f46bf3740 20 client.15399 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.848+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f20a38e0 0x5576f24a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.848+0000 7f3f46bf3740 4 client.15399 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.848+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f2404ca0 con 0x5576f20a38e0 2024-09-21T01:22:08.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.848+0000 7f3f3d400640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f20a38e0 0x5576f24a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.849+0000 7f3f3d400640 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f20a38e0 0x5576f24a95f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a767fc749a5698f server_cookie=82cdab4d0e1e3cf9 in_seq=0 out_seq=0 2024-09-21T01:22:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.849+0000 7f3f1ea00640 10 client.15399 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f1ea00640 1 -- 172.21.15.27:0/742162808 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576f2404ca0 con 0x5576f20a38e0 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f1ea00640 10 client.15399 handle_command_reply: tid=0 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 client.15399 shutdown 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 msgr2=0x5576f2497720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f2497720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c6a0 tx=0x7f3f240acf10 comp rx=0 tx=0).stop 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 shutdown_connections 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20e4f60 0x5576f24a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f2497720 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f20a38e0 0x5576f24a95f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/742162808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 >> 172.21.15.27:0/742162808 conn(0x5576f20d9350 msgr2=0x5576f248e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f1ea00640 0 client.15399 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 shutdown_connections 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 1 -- 172.21.15.27:0/742162808 wait complete. 2024-09-21T01:22:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.850+0000 7f3f46bf3740 20 client.15399 trim_cache size 0 max 16384 2024-09-21T01:22:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.869+0000 7f3f46bf3740 1 Processor -- start 2024-09-21T01:22:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.869+0000 7f3f46bf3740 1 -- start start 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f24a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f24a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f24a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5576f24a7930 con 0x5576f20c4280 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5576f24a8000 con 0x5576f20e4f60 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5576f24a86d0 con 0x5576f20bc0a0 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f24a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f24a95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32978/0 (socket says 172.21.15.27:32978) 2024-09-21T01:22:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1797747459 learned_addr learned my addr 172.21.15.27:0/1797747459 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3de00640 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f24a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3d400640 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f24a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1797747459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 msgr2=0x5576f24a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f24a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1797747459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 msgr2=0x5576f24a3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f24a3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.870+0000 7f3f3ca00640 1 -- 172.21.15.27:0/1797747459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f24a7760 con 0x5576f20c4280 2024-09-21T01:22:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.871+0000 7f3f3d400640 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f24a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:22:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.871+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f24a95f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f240aab50 tx=0x7f3f240a9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be00cd2d5545e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.871+0000 7f3f29c00640 1 -- 172.21.15.27:0/1797747459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240adfe0 con 0x5576f20c4280 2024-09-21T01:22:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.871+0000 7f3f29c00640 1 -- 172.21.15.27:0/1797747459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f240b7300 con 0x5576f20c4280 2024-09-21T01:22:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.871+0000 7f3f29c00640 1 -- 172.21.15.27:0/1797747459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240b6840 con 0x5576f20c4280 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1797747459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 msgr2=0x5576f24a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f24a95f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f240aab50 tx=0x7f3f240a9780 comp rx=0 tx=0).stop 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1797747459 shutdown_connections 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20e4f60 0x5576f24a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20c4280 0x5576f24a95f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/1797747459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20bc0a0 0x5576f24a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1797747459 >> 172.21.15.27:0/1797747459 conn(0x5576f20d9350 msgr2=0x5576f24842c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1797747459 shutdown_connections 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.872+0000 7f3f46bf3740 1 -- 172.21.15.27:0/1797747459 wait complete. 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 -- start start 2024-09-21T01:22:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 Processor -- start 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20c4280 0x5576f24ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20e4f60 0x5576f24ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5576f24a9470 con 0x5576f20e4f60 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5576f24a8da0 con 0x5576f20bc0a0 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.873+0000 7f3f46bf3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5576f24a9b40 con 0x5576f20c4280 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f46bf3740 10 client.0 _ll_register_callbacks cb 0x7f3f240a9850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45734/0 (socket says 172.21.15.27:45734) 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 -- 172.21.15.27:0/3685471331 learned_addr learned my addr 172.21.15.27:0/3685471331 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20c4280 0x5576f24ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3de00640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20e4f60 0x5576f24ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 -- 172.21.15.27:0/3685471331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20c4280 msgr2=0x5576f24ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20c4280 0x5576f24ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 -- 172.21.15.27:0/3685471331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20e4f60 msgr2=0x5576f24ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20e4f60 0x5576f24ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.874+0000 7f3f3d400640 1 -- 172.21.15.27:0/3685471331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f24abcf0 con 0x5576f20bc0a0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f3d400640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24b1dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5576f24a9cf0 tx=0x7f3f2c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b9dc582b7b6789 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f1ea00640 1 -- 172.21.15.27:0/3685471331 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c036060 con 0x5576f20bc0a0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f1ea00640 1 -- 172.21.15.27:0/3685471331 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3f2c004320 con 0x5576f20bc0a0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f240b7300 con 0x5576f20bc0a0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f1ea00640 1 -- 172.21.15.27:0/3685471331 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f2c03d050 con 0x5576f20bc0a0 2024-09-21T01:22:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f46bf3740 10 client.25445 fetch_fsmap learned FSMap version 28 2024-09-21T01:22:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.875+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576f24b1690 con 0x5576f20bc0a0 2024-09-21T01:22:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f1ea00640 1 -- 172.21.15.27:0/3685471331 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f240b7300 con 0x5576f20bc0a0 2024-09-21T01:22:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f1ea00640 1 -- 172.21.15.27:0/3685471331 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3f2c031640 con 0x5576f20bc0a0 2024-09-21T01:22:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f46bf3740 10 client.25445 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T01:22:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f46bf3740 10 client.25445 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:22:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f46bf3740 20 client.25445 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f2078c90 0x5576f2518020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f46bf3740 4 client.25445 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "status"}) -- 0x5576f2400750 con 0x5576f2078c90 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.876+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f2078c90 0x5576f2518020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f3ca00640 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f2078c90 0x5576f2518020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5889e9ea2a91feb4 server_cookie=18d1c06602162a98 in_seq=0 out_seq=0 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f1ea00640 10 client.25445 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f1ea00640 1 -- 172.21.15.27:0/3685471331 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5576f2400750 con 0x5576f2078c90 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f1ea00640 10 client.25445 handle_command_reply: tid=0 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 client.25445 shutdown 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 msgr2=0x5576f24b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24b1dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5576f24a9cf0 tx=0x7f3f2c0046c0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 shutdown_connections 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5576f20e4f60 0x5576f24ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5576f20c4280 0x5576f24ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5576f2078c90 0x5576f2518020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 --2- 172.21.15.27:0/3685471331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5576f20bc0a0 0x5576f24b1dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.877+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 >> 172.21.15.27:0/3685471331 conn(0x5576f20d9350 msgr2=0x5576f24842c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f1ea00640 0 client.25445 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 shutdown_connections 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f46bf3740 1 -- 172.21.15.27:0/3685471331 wait complete. 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f46bf3740 20 client.25445 trim_cache size 0 max 16384 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24776, 2024-09-21T01:22:08.881 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 180.14748737799999, 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 28, 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 88, 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 194.65542654800001 2024-09-21T01:22:08.882 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f200b6560 msgr2=0x7f3f200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f200b6560 0x7f3f200b8a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5576f248a2f0 tx=0x7f3f2c009260 comp rx=0 tx=0).stop 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b510 msgr2=0x7f3f3819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.878+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b510 0x7f3f3819ca30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3f3806c860 tx=0x7f3f2400ee50 comp rx=0 tx=0).stop 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 shutdown_connections 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f3f200b6560 0x7f3f200b8a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3f3810be60 0x7f3f381a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f3810b510 0x7f3f3819ca30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 --2- 172.21.15.27:0/1094380911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f38105bd0 0x7f3f3819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 >> 172.21.15.27:0/1094380911 conn(0x7f3f38101620 msgr2=0x7f3f38103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:08.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 shutdown_connections 2024-09-21T01:22:08.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:08.879+0000 7f3f3e800640 1 -- 172.21.15.27:0/1094380911 wait complete. 2024-09-21T01:22:09.803 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T01:22:09.824 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T01:22:09.824 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T01:22:09.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:09.825 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T01:22:09.861 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T01:22:09.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:09.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T01:22:09.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:09 smithi098 ceph-mon[34372]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:09.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:09 smithi027 ceph-mon[30375]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:09.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:09.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T01:22:09.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:09 smithi089 ceph-mon[34486]: pgmap v392: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:10.280 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-21T01:22:10.280 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T01:22:10.280 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:10.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T01:22:10.669 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 -- 172.21.15.27:0/105172058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 msgr2=0x7efdbc116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 --2- 172.21.15.27:0/105172058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc116b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009ca0 tx=0x7efdb402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 -- 172.21.15.27:0/105172058 shutdown_connections 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 --2- 172.21.15.27:0/105172058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc116b60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 --2- 172.21.15.27:0/105172058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 0x7efdbc114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 --2- 172.21.15.27:0/105172058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 -- 172.21.15.27:0/105172058 >> 172.21.15.27:0/105172058 conn(0x7efdbc101810 msgr2=0x7efdbc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.369+0000 7efdc3400640 1 -- 172.21.15.27:0/105172058 shutdown_connections 2024-09-21T01:22:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.370+0000 7efdc3400640 1 -- 172.21.15.27:0/105172058 wait complete. 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 Processor -- start 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 -- start start 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 0x7efdbc1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc1a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc11a100 con 0x7efdbc114740 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc119f80 con 0x7efdbc078290 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.371+0000 7efdc3400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efdbc11a280 con 0x7efdbc077940 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdbbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 0x7efdbc1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc1a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdbbe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 0x7efdbc1a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45750/0 (socket says 172.21.15.27:45750) 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc1a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:45904/0 (socket says 172.21.15.27:45904) 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdbbe00640 1 -- 172.21.15.27:0/2609088493 learned_addr learned my addr 172.21.15.27:0/2609088493 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 -- 172.21.15.27:0/2609088493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 msgr2=0x7efdbc1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 0x7efdbc1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 -- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 msgr2=0x7efdbc1a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc1a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc0a00640 1 -- 172.21.15.27:0/2609088493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdbc1a69a0 con 0x7efdbc077940 2024-09-21T01:22:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.372+0000 7efdc1400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc1a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.373+0000 7efdc0a00640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc1a0a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdb0005fd0 tx=0x7efdb000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.373+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0009c30 con 0x7efdbc077940 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.373+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdbc1a6c90 con 0x7efdbc077940 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.373+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb0009dd0 con 0x7efdbc077940 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.373+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0010460 con 0x7efdbc077940 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.373+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdbc1ad3f0 con 0x7efdbc077940 2024-09-21T01:22:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.374+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdbc077d50 con 0x7efdbc077940 2024-09-21T01:22:11.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.377+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7efdb0003e80 con 0x7efdbc077940 2024-09-21T01:22:11.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.379+0000 7efdb9600640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efda40b6560 0x7efda40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.379+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efdb00e7780 con 0x7efdbc077940 2024-09-21T01:22:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.379+0000 7efdbbe00640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efda40b6560 0x7efda40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.380+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efdb0017020 con 0x7efdbc077940 2024-09-21T01:22:11.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.380+0000 7efdbbe00640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efda40b6560 0x7efda40b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efdbc079d30 tx=0x7efda8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:11.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.621+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efdbc02e640 con 0x7efdbc077940 2024-09-21T01:22:11.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.628+0000 7efdb9600640 1 -- 172.21.15.27:0/2609088493 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7efdb00b1f00 con 0x7efdbc077940 2024-09-21T01:22:11.629 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efda40b6560 msgr2=0x7efda40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efda40b6560 0x7efda40b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efdbc079d30 tx=0x7efda8005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 msgr2=0x7efdbc1a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc1a0a30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdb0005fd0 tx=0x7efdb000dde0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 shutdown_connections 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efda40b6560 0x7efda40b8a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdbc114740 0x7efdbc1a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efdbc078290 0x7efdbc1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 --2- 172.21.15.27:0/2609088493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efdbc077940 0x7efdbc1a0a30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 >> 172.21.15.27:0/2609088493 conn(0x7efdbc101810 msgr2=0x7efdbc116700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 shutdown_connections 2024-09-21T01:22:11.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:11.631+0000 7efdc3400640 1 -- 172.21.15.27:0/2609088493 wait complete. 2024-09-21T01:22:11.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:11 smithi098 ceph-mon[34372]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:22:11.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:11 smithi027 ceph-mon[30375]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:22:11.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:11 smithi089 ceph-mon[34486]: pgmap v393: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:22:12.519 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T01:22:12.519 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T01:22:12.519 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-09-21T01:22:12.520 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T01:22:12.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:12.520 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T01:22:12.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T01:22:12.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T01:22:12.520 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T01:22:12.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:12.520 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-09-21T01:22:12.557 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-09-21T01:22:12.558 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41954sec preferred_lft 41954sec 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-09-21T01:22:12.559 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-09-21T01:22:12.559 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T01:22:12.560 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:22:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:22:12.696 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:12 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2609088493' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:22:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:12.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:12.789 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T01:22:12.849 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-09-21T01:22:12.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:12.852 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-09-21T01:22:12.889 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-09-21T01:22:12.889 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T01:22:12.890 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T01:22:12.890 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T01:22:12.890 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T01:22:12.890 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:22:12.890 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:22:12.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T01:22:12.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T01:22:12.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T01:22:12.890 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:22:12.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:12 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2609088493' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:22:12.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:12 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2609088493' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T01:22:13.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:22:13.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:13.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:13.092 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T01:22:13.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:13.129 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-09-21T01:22:13.205 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:22:13.205 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:22:13.205 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T01:22:13.205 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T01:22:13.206 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:22:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:22:13.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:13.385 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T01:22:13.386 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:22:13.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:22:13.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:13.610 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T01:22:13.610 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-09-21T01:22:13.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-09-21T01:22:13.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T01:22:13.610 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-09-21T01:22:13.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:22:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:13.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T01:22:13.782 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:13.818 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T01:22:13.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T01:22:13.818 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:13.893 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T01:22:13.894 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T01:22:13.894 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/467e0d3e-aecf-42d9-88a8-be95e03ed056 2024-09-21T01:22:13.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:13.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/467e0d3e-aecf-42d9-88a8-be95e03ed056 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-21T01:22:13.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:13 smithi098 ceph-mon[34372]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:13.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:13 smithi027 ceph-mon[30375]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:14.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:13 smithi089 ceph-mon[34486]: pgmap v394: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:22:14.093 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-21T01:22:14.093 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-21T01:22:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-21T01:22:14.732 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-21T01:22:14.732 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:6789,172.21.15.89:6789,172.21.15.98:6789" 2024-09-21T01:22:14.732 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@fe42ec90-77b5-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/467e0d3e-aecf-42d9-88a8-be95e03ed056 2024-09-21T01:22:14.732 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.27:6789/172.21.15.89:6789/172.21.15.98:6789" will pass to kernel 2024-09-21T01:22:14.734 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T01:22:14.734 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-09-21T01:22:14.734 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T01:22:14.734 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T01:22:14.734 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T01:22:14.737 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T01:22:14.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:22:14.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:14.798 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T01:22:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:22:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:14.893 INFO:teuthology.orchestra.run.smithi027.stdout:client25441 2024-09-21T01:22:14.893 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:22:14.893 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T01:22:14.929 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T01:22:15.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:22:15.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:15.062 INFO:teuthology.orchestra.run.smithi027.stdout:client25441 2024-09-21T01:22:15.063 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T01:22:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T01:22:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:15.158 INFO:teuthology.orchestra.run.smithi027.stdout:client25441 2024-09-21T01:22:15.158 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490.client25441/status') 2024-09-21T01:22:15.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T01:22:15.262 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.25441 (3)172.21.15.27:0/574566687 2024-09-21T01:22:15.262 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-09-21T01:22:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-09-21T01:22:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-09-21T01:22:15.263 INFO:teuthology.orchestra.run.smithi027.stderr:70 bytes copied, 7.2711e-05 s, 963 kB/s 2024-09-21T01:22:15.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T01:22:15.274 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T01:22:15.305 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T01:22:15.305 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:22:15.305 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T01:22:15.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.536+0000 7f6921e00640 1 -- 172.21.15.27:0/4089585652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c104920 msgr2=0x7f691c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.536+0000 7f6921e00640 1 --2- 172.21.15.27:0/4089585652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c104920 0x7f691c104d30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f690c009860 tx=0x7f690c02a340 comp rx=0 tx=0).stop 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 -- 172.21.15.27:0/4089585652 shutdown_connections 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 --2- 172.21.15.27:0/4089585652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 0x7f691c110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 --2- 172.21.15.27:0/4089585652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c105270 0x7f691c10d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 --2- 172.21.15.27:0/4089585652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c104920 0x7f691c104d30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 -- 172.21.15.27:0/4089585652 >> 172.21.15.27:0/4089585652 conn(0x7f691c07a710 msgr2=0x7f691c07ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 -- 172.21.15.27:0/4089585652 shutdown_connections 2024-09-21T01:22:15.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.537+0000 7f6921e00640 1 -- 172.21.15.27:0/4089585652 wait complete. 2024-09-21T01:22:15.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.538+0000 7f6921e00640 1 Processor -- start 2024-09-21T01:22:15.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.538+0000 7f6921e00640 1 -- start start 2024-09-21T01:22:15.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.538+0000 7f6921e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c104920 0x7f691c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.538+0000 7f6921e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c105270 0x7f691c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.538+0000 7f6921e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 0x7f691c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6921e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f691c113a00 con 0x7f691c105270 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6921e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f691c113880 con 0x7f691c104920 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6921e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f691c113b80 con 0x7f691c10dd80 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f691aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c104920 0x7f691c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6913e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c105270 0x7f691c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f691aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c104920 0x7f691c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:45760/0 (socket says 172.21.15.27:45760) 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f691aa00640 1 -- 172.21.15.27:0/2579608358 learned_addr learned my addr 172.21.15.27:0/2579608358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:15.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f691b400640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 0x7f691c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:15.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6913e00640 1 -- 172.21.15.27:0/2579608358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 msgr2=0x7f691c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:15.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6913e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 0x7f691c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:15.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6913e00640 1 -- 172.21.15.27:0/2579608358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c104920 msgr2=0x7f691c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:15.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6913e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c104920 0x7f691c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:15.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f6913e00640 1 -- 172.21.15.27:0/2579608358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c1a03b0 con 0x7f691c105270 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.539+0000 7f691b400640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 0x7f691c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.540+0000 7f6913e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c105270 0x7f691c19aa20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f691c06b500 tx=0x7f690400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.540+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690400cc90 con 0x7f691c105270 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.540+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1a06a0 con 0x7f691c105270 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.540+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f690400ee70 con 0x7f691c105270 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.540+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904018680 con 0x7f691c105270 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.540+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a6e30 con 0x7f691c105270 2024-09-21T01:22:15.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.541+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c06c050 con 0x7f691c105270 2024-09-21T01:22:15.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.543+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f69040188e0 con 0x7f691c105270 2024-09-21T01:22:15.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.545+0000 7f6920a00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69000b6560 0x7f69000b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:15.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.545+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f69040e7a60 con 0x7f691c105270 2024-09-21T01:22:15.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.545+0000 7f691aa00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69000b6560 0x7f69000b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:15.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.546+0000 7f691aa00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69000b6560 0x7f69000b8a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f690c0057c0 tx=0x7f690c030340 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:15.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.549+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69040b21e0 con 0x7f691c105270 2024-09-21T01:22:15.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:15.794+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f691c19b5a0 con 0x7f691c105270 2024-09-21T01:22:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:15 smithi098 ceph-mon[34372]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:22:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:22:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:22:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:22:15.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:22:15.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:15 smithi027 ceph-mon[30375]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:22:15.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:22:15.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:22:15.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:22:15.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:22:15.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:15 smithi089 ceph-mon[34486]: pgmap v395: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:22:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:22:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:22:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:22:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:22:16.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.648+0000 7f6920a00640 1 -- 172.21.15.27:0/2579608358 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f691c19b5a0 con 0x7f691c105270 2024-09-21T01:22:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.651+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69000b6560 msgr2=0x7f69000b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.651+0000 7f6921e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69000b6560 0x7f69000b8a70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f690c0057c0 tx=0x7f690c030340 comp rx=0 tx=0).stop 2024-09-21T01:22:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.651+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c105270 msgr2=0x7f691c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.651+0000 7f6921e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c105270 0x7f691c19aa20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f691c06b500 tx=0x7f690400ec20 comp rx=0 tx=0).stop 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 shutdown_connections 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f69000b6560 0x7f69000b8a70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f691c10dd80 0x7f691c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f691c105270 0x7f691c19aa20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 --2- 172.21.15.27:0/2579608358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f691c104920 0x7f691c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 >> 172.21.15.27:0/2579608358 conn(0x7f691c07a710 msgr2=0x7f691c107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 shutdown_connections 2024-09-21T01:22:16.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.652+0000 7f6921e00640 1 -- 172.21.15.27:0/2579608358 wait complete. 2024-09-21T01:22:16.679 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T01:22:16.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:16 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2579608358' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T01:22:16.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 -- 172.21.15.27:0/3722266616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740760a0 msgr2=0x7fef7410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 --2- 172.21.15.27:0/3722266616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740760a0 0x7fef7410f990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fef60009900 tx=0x7fef6002a410 comp rx=0 tx=0).stop 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 -- 172.21.15.27:0/3722266616 shutdown_connections 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 --2- 172.21.15.27:0/3722266616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 --2- 172.21.15.27:0/3722266616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef740760a0 0x7fef7410f990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 --2- 172.21.15.27:0/3722266616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef74075750 0x7fef74075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 -- 172.21.15.27:0/3722266616 >> 172.21.15.27:0/3722266616 conn(0x7fef740ff320 msgr2=0x7fef74101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 -- 172.21.15.27:0/3722266616 shutdown_connections 2024-09-21T01:22:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.909+0000 7fef7a800640 1 -- 172.21.15.27:0/3722266616 wait complete. 2024-09-21T01:22:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.910+0000 7fef7a800640 1 Processor -- start 2024-09-21T01:22:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.910+0000 7fef7a800640 1 -- start start 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.910+0000 7fef7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74075750 0x7fef7419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.910+0000 7fef7a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef740760a0 0x7fef7419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.910+0000 7fef7a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.910+0000 7fef7a800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fef74115a90 con 0x7fef74075750 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef7a800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fef74115910 con 0x7fef740760a0 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef7a800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef74115c10 con 0x7fef7410fed0 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74075750 0x7fef7419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef78a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef740760a0 0x7fef7419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74075750 0x7fef7419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42862/0 (socket says 172.21.15.27:42862) 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef78a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef740760a0 0x7fef7419c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59644/0 (socket says 172.21.15.27:59644) 2024-09-21T01:22:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:36676/0 (socket says 172.21.15.27:36676) 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79400640 1 -- 172.21.15.27:0/835926850 learned_addr learned my addr 172.21.15.27:0/835926850 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 -- 172.21.15.27:0/835926850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef740760a0 msgr2=0x7fef7419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef740760a0 0x7fef7419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 -- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74075750 msgr2=0x7fef7419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74075750 0x7fef7419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.911+0000 7fef79e00640 1 -- 172.21.15.27:0/835926850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef741a2360 con 0x7fef7410fed0 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.912+0000 7fef79e00640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741a1c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009fd0 tx=0x7fef6c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.912+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00ce20 con 0x7fef7410fed0 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.912+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef741a2650 con 0x7fef7410fed0 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.912+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef6c005020 con 0x7fef7410fed0 2024-09-21T01:22:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.912+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c010690 con 0x7fef7410fed0 2024-09-21T01:22:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.912+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef741a8de0 con 0x7fef7410fed0 2024-09-21T01:22:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.913+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef74075b60 con 0x7fef7410fed0 2024-09-21T01:22:16.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.915+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fef6c0054d0 con 0x7fef7410fed0 2024-09-21T01:22:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.917+0000 7fef72000640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fef5c0b6550 0x7fef5c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.917+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fef6c0e7f10 con 0x7fef7410fed0 2024-09-21T01:22:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.917+0000 7fef79400640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fef5c0b6550 0x7fef5c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:16.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.918+0000 7fef79400640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fef5c0b6550 0x7fef5c0b8a60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef68009c90 tx=0x7fef68005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:16.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:16.919+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef6c0ae2b0 con 0x7fef7410fed0 2024-09-21T01:22:16.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:16 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2579608358' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T01:22:16.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:16 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2579608358' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T01:22:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.150+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fef7410a0f0 con 0x7fef5c0b6550 2024-09-21T01:22:17.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.172+0000 7fef72000640 1 -- 172.21.15.27:0/835926850 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fef7410a0f0 con 0x7fef5c0b6550 2024-09-21T01:22:17.172 INFO:teuthology.orchestra.run.smithi027.stdout:{} 2024-09-21T01:22:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fef5c0b6550 msgr2=0x7fef5c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fef5c0b6550 0x7fef5c0b8a60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef68009c90 tx=0x7fef68005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 msgr2=0x7fef741a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741a1c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009fd0 tx=0x7fef6c00ef10 comp rx=0 tx=0).stop 2024-09-21T01:22:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 shutdown_connections 2024-09-21T01:22:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fef5c0b6550 0x7fef5c0b8a60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef7410fed0 0x7fef741a1c60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fef740760a0 0x7fef7419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 --2- 172.21.15.27:0/835926850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef74075750 0x7fef7419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.174+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 >> 172.21.15.27:0/835926850 conn(0x7fef740ff320 msgr2=0x7fef741099e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:17.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.175+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 shutdown_connections 2024-09-21T01:22:17.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.175+0000 7fef7a800640 1 -- 172.21.15.27:0/835926850 wait complete. 2024-09-21T01:22:17.201 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T01:22:17.212 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T01:22:17.220 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T01:22:17.220 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:22:17.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T01:22:17.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:22:17.156+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:22:17.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:22:17.156+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:22:17.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:22:17.156+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:22:17.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:22:17.156+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:22:17.431 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:22:17.156+0000 7f655a200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:22:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.447+0000 7f2d32800640 1 -- 172.21.15.27:0/1278518867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 msgr2=0x7f2d2c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.447+0000 7f2d32800640 1 --2- 172.21.15.27:0/1278518867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c1100a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009ca0 tx=0x7f2d2402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 -- 172.21.15.27:0/1278518867 shutdown_connections 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 --2- 172.21.15.27:0/1278518867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c1100a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 --2- 172.21.15.27:0/1278518867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 --2- 172.21.15.27:0/1278518867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c104750 0x7f2d2c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 -- 172.21.15.27:0/1278518867 >> 172.21.15.27:0/1278518867 conn(0x7f2d2c0fe200 msgr2=0x7f2d2c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 -- 172.21.15.27:0/1278518867 shutdown_connections 2024-09-21T01:22:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.448+0000 7f2d32800640 1 -- 172.21.15.27:0/1278518867 wait complete. 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 Processor -- start 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 -- start start 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c104750 0x7f2d2c102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c113820 con 0x7f2d2c10dbb0 2024-09-21T01:22:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1136a0 con 0x7f2d2c104750 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.449+0000 7f2d32800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1139a0 con 0x7f2d2c1050a0 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d30a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d30a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c1029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:36718/0 (socket says 172.21.15.27:36718) 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c104750 0x7f2d2c102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d30a00640 1 -- 172.21.15.27:0/2053040477 learned_addr learned my addr 172.21.15.27:0/2053040477 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 -- 172.21.15.27:0/2053040477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 msgr2=0x7f2d2c1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 -- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 msgr2=0x7f2d2c102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 -- 172.21.15.27:0/2053040477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1a84b0 con 0x7f2d2c104750 2024-09-21T01:22:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31e00640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d30a00640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.450+0000 7f2d31400640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c104750 0x7f2d2c102480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e9f0 tx=0x7f2d2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.451+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000ccc0 con 0x7f2d2c104750 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.451+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c1a87a0 con 0x7f2d2c104750 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.451+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d200108e0 con 0x7f2d2c104750 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.451+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20018880 con 0x7f2d2c104750 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.451+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c1a8d40 con 0x7f2d2c104750 2024-09-21T01:22:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.452+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c104b60 con 0x7f2d2c104750 2024-09-21T01:22:17.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.455+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7f2d20010430 con 0x7f2d2c104750 2024-09-21T01:22:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.456+0000 7f2d2a000640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2d140b6580 0x7f2d140b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.456+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f2d200e7fd0 con 0x7f2d2c104750 2024-09-21T01:22:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.457+0000 7f2d30a00640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2d140b6580 0x7f2d140b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.457+0000 7f2d30a00640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2d140b6580 0x7f2d140b8a90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c110090 tx=0x7f2d18005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:17.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.458+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2d200b2750 con 0x7f2d2c104750 2024-09-21T01:22:17.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.678+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2d2c106ca0 con 0x7f2d140b6580 2024-09-21T01:22:17.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.858+0000 7f2d2a000640 1 -- 172.21.15.27:0/2053040477 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2d2c106ca0 con 0x7f2d140b6580 2024-09-21T01:22:17.858 INFO:teuthology.orchestra.run.smithi027.stdout:Schedule set for path / 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2d140b6580 msgr2=0x7f2d140b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2d140b6580 0x7f2d140b8a90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c110090 tx=0x7f2d18005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c104750 msgr2=0x7f2d2c102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c104750 0x7f2d2c102480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e9f0 tx=0x7f2d2000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 shutdown_connections 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2d140b6580 0x7f2d140b8a90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d2c10dbb0 0x7f2d2c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d2c1050a0 0x7f2d2c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 --2- 172.21.15.27:0/2053040477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d2c104750 0x7f2d2c102480 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 >> 172.21.15.27:0/2053040477 conn(0x7f2d2c0fe200 msgr2=0x7f2d2c105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:17.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 shutdown_connections 2024-09-21T01:22:17.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:17.860+0000 7f2d32800640 1 -- 172.21.15.27:0/2053040477 wait complete. 2024-09-21T01:22:17.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T01:22:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-mon[30375]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T01:22:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2579608358' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T01:22:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-mon[30375]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:22:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:17 smithi027 ceph-mon[30375]: from='client.25453 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:17 smithi089 ceph-mon[34486]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T01:22:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:17 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2579608358' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T01:22:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:17 smithi089 ceph-mon[34486]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:22:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:17 smithi089 ceph-mon[34486]: from='client.25453 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 -- 172.21.15.27:0/1382068931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb160105b10 msgr2=0x7fb160105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 --2- 172.21.15.27:0/1382068931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb160105b10 0x7fb160105f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb150009ca0 tx=0x7fb15002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 -- 172.21.15.27:0/1382068931 shutdown_connections 2024-09-21T01:22:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 --2- 172.21.15.27:0/1382068931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb16010bd90 0x7fb1601125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 --2- 172.21.15.27:0/1382068931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb16010b440 0x7fb16010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 --2- 172.21.15.27:0/1382068931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb160105b10 0x7fb160105f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 -- 172.21.15.27:0/1382068931 >> 172.21.15.27:0/1382068931 conn(0x7fb160101510 msgr2=0x7fb160103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 -- 172.21.15.27:0/1382068931 shutdown_connections 2024-09-21T01:22:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.114+0000 7fb164a00640 1 -- 172.21.15.27:0/1382068931 wait complete. 2024-09-21T01:22:18.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.115+0000 7fb164a00640 1 Processor -- start 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.115+0000 7fb164a00640 1 -- start start 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb164a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 0x7fb16019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb164a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16010b440 0x7fb16019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb164a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb16010bd90 0x7fb1601a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb164a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb160115aa0 con 0x7fb160105b10 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb164a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb160115920 con 0x7fb16010bd90 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb164a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb160115c20 con 0x7fb16010b440 2024-09-21T01:22:18.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb15e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 0x7fb16019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb15e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 0x7fb16019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42918/0 (socket says 172.21.15.27:42918) 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb15e800640 1 -- 172.21.15.27:0/2369860409 learned_addr learned my addr 172.21.15.27:0/2369860409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb155e00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16010b440 0x7fb16019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.116+0000 7fb15f200640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb16010bd90 0x7fb1601a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb155e00640 1 -- 172.21.15.27:0/2369860409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb16010bd90 msgr2=0x7fb1601a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb155e00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb16010bd90 0x7fb1601a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb155e00640 1 -- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 msgr2=0x7fb16019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb155e00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 0x7fb16019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb155e00640 1 -- 172.21.15.27:0/2369860409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1601a23b0 con 0x7fb16010b440 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb15e800640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 0x7fb16019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb155e00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16010b440 0x7fb16019ca20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb16006c770 tx=0x7fb14800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14800cd00 con 0x7fb16010b440 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1480035f0 con 0x7fb16010b440 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148010690 con 0x7fb16010b440 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1601a26a0 con 0x7fb16010b440 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.117+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1601a8e30 con 0x7fb16010b440 2024-09-21T01:22:18.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.118+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb160105f20 con 0x7fb16010b440 2024-09-21T01:22:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.121+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 154529+0+0 (secure 0 0 0) 0x7fb1480037a0 con 0x7fb16010b440 2024-09-21T01:22:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.122+0000 7fb157e00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb1440b65d0 0x7fb1440b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.123+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb1480e7da0 con 0x7fb16010b440 2024-09-21T01:22:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.123+0000 7fb15e800640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb1440b65d0 0x7fb1440b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.123+0000 7fb15e800640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb1440b65d0 0x7fb1440b8ae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb150000f80 tx=0x7fb150005d80 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.124+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1480ae140 con 0x7fb16010b440 2024-09-21T01:22:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:17 smithi098 ceph-mon[34372]: pgmap v396: 129 pgs: 129 active+clean; 649 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T01:22:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:17 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2579608358' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T01:22:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:17 smithi098 ceph-mon[34372]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T01:22:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:17 smithi098 ceph-mon[34372]: from='client.25453 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:18.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.351+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb160063800 con 0x7fb1440b65d0 2024-09-21T01:22:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.424+0000 7fb157e00640 1 -- 172.21.15.27:0/2369860409 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb160063800 con 0x7fb1440b65d0 2024-09-21T01:22:18.425 INFO:teuthology.orchestra.run.smithi027.stdout:Retention added to path / 2024-09-21T01:22:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb1440b65d0 msgr2=0x7fb1440b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb1440b65d0 0x7fb1440b8ae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb150000f80 tx=0x7fb150005d80 comp rx=0 tx=0).stop 2024-09-21T01:22:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16010b440 msgr2=0x7fb16019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16010b440 0x7fb16019ca20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb16006c770 tx=0x7fb14800ee50 comp rx=0 tx=0).stop 2024-09-21T01:22:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 shutdown_connections 2024-09-21T01:22:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb1440b65d0 0x7fb1440b8ae0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb16010bd90 0x7fb1601a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16010b440 0x7fb16019ca20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 --2- 172.21.15.27:0/2369860409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb160105b10 0x7fb16019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.426+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 >> 172.21.15.27:0/2369860409 conn(0x7fb160101510 msgr2=0x7fb16010aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.427+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 shutdown_connections 2024-09-21T01:22:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.427+0000 7fb164a00640 1 -- 172.21.15.27:0/2369860409 wait complete. 2024-09-21T01:22:18.454 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T01:22:18.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 -- 172.21.15.27:0/3846512265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f683810dba0 msgr2=0x7f6838110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 --2- 172.21.15.27:0/3846512265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f683810dba0 0x7f6838110090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6830009be0 tx=0x7f683002a6c0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 -- 172.21.15.27:0/3846512265 shutdown_connections 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 --2- 172.21.15.27:0/3846512265 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f683810dba0 0x7f6838110090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 --2- 172.21.15.27:0/3846512265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838105090 0x7f683810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 --2- 172.21.15.27:0/3846512265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6838104740 0x7f6838104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 -- 172.21.15.27:0/3846512265 >> 172.21.15.27:0/3846512265 conn(0x7f68380fe210 msgr2=0x7f6838100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 -- 172.21.15.27:0/3846512265 shutdown_connections 2024-09-21T01:22:18.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.733+0000 7f6837200640 1 -- 172.21.15.27:0/3846512265 wait complete. 2024-09-21T01:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.734+0000 7f6837200640 1 Processor -- start 2024-09-21T01:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 -- start start 2024-09-21T01:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838104740 0x7f683819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 0x7f683819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683810dba0 0x7f68381a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6838113820 con 0x7f6838104740 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f68381136a0 con 0x7f6838105090 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6837200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f68381139a0 con 0x7f683810dba0 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 0x7f683819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6835e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838104740 0x7f683819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 0x7f683819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59724/0 (socket says 172.21.15.27:59724) 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6835e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838104740 0x7f683819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42938/0 (socket says 172.21.15.27:42938) 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6835400640 1 -- 172.21.15.27:0/23221798 learned_addr learned my addr 172.21.15.27:0/23221798 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.735+0000 7f6836800640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683810dba0 0x7f68381a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f6835400640 1 -- 172.21.15.27:0/23221798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683810dba0 msgr2=0x7f68381a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f6835400640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683810dba0 0x7f68381a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f6835400640 1 -- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838104740 msgr2=0x7f683819c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f6835400640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838104740 0x7f683819c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f6835400640 1 -- 172.21.15.27:0/23221798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68381a2310 con 0x7f6838105090 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f6835400640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 0x7f683819c980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f682400e9f0 tx=0x7f682400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682400ccc0 con 0x7f6838105090 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68240108e0 con 0x7f6838105090 2024-09-21T01:22:18.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.736+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824018880 con 0x7f6838105090 2024-09-21T01:22:18.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.737+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68381a2600 con 0x7f6838105090 2024-09-21T01:22:18.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.737+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68381a8d90 con 0x7f6838105090 2024-09-21T01:22:18.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.738+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683806b450 con 0x7f6838105090 2024-09-21T01:22:18.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.741+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f6824010430 con 0x7f6838105090 2024-09-21T01:22:18.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.742+0000 7f682ea00640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f681c0b66f0 0x7f681c0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:18.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.742+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f68240e8270 con 0x7f6838105090 2024-09-21T01:22:18.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.743+0000 7f6835e00640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f681c0b66f0 0x7f681c0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:18.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.743+0000 7f6835e00640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f681c0b66f0 0x7f681c0b8c00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6828005fd0 tx=0x7f6828009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:18.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.743+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68240ae3f0 con 0x7f6838105090 2024-09-21T01:22:18.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:18 smithi027 ceph-mon[30375]: from='client.25475 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.967+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f683810d270 con 0x7f681c0b66f0 2024-09-21T01:22:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.975+0000 7f682ea00640 1 -- 172.21.15.27:0/23221798 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f683810d270 con 0x7f681c0b66f0 2024-09-21T01:22:18.975 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T01:22:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f681c0b66f0 msgr2=0x7f681c0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f681c0b66f0 0x7f681c0b8c00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6828005fd0 tx=0x7f6828009260 comp rx=0 tx=0).stop 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 msgr2=0x7f683819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 0x7f683819c980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f682400e9f0 tx=0x7f682400edf0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 shutdown_connections 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f681c0b66f0 0x7f681c0b8c00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683810dba0 0x7f68381a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6838105090 0x7f683819c980 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 --2- 172.21.15.27:0/23221798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838104740 0x7f683819c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 >> 172.21.15.27:0/23221798 conn(0x7f68380fe210 msgr2=0x7f683810cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 shutdown_connections 2024-09-21T01:22:18.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:18.977+0000 7f6837200640 1 -- 172.21.15.27:0/23221798 wait complete. 2024-09-21T01:22:19.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T01:22:19.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:18 smithi089 ceph-mon[34486]: from='client.25475 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:19.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:18 smithi098 ceph-mon[34372]: from='client.25475 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 -- 172.21.15.27:0/845783279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e810bf60 msgr2=0x7f54e8112790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 --2- 172.21.15.27:0/845783279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e810bf60 0x7f54e8112790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009ca0 tx=0x7f54e002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 -- 172.21.15.27:0/845783279 shutdown_connections 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 --2- 172.21.15.27:0/845783279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e810bf60 0x7f54e8112790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 --2- 172.21.15.27:0/845783279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 --2- 172.21.15.27:0/845783279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e8105ce0 0x7f54e81060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 -- 172.21.15.27:0/845783279 >> 172.21.15.27:0/845783279 conn(0x7f54e81016c0 msgr2=0x7f54e8103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:19.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 -- 172.21.15.27:0/845783279 shutdown_connections 2024-09-21T01:22:19.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.229+0000 7f54ef400640 1 -- 172.21.15.27:0/845783279 wait complete. 2024-09-21T01:22:19.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 Processor -- start 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 -- start start 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e8105ce0 0x7f54e819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e810bf60 0x7f54e81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8115c20 con 0x7f54e8105ce0 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8115aa0 con 0x7f54e810bf60 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.230+0000 7f54ef400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8115da0 con 0x7f54e810b610 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54e7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e8105ce0 0x7f54e819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54e7e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:36780/0 (socket says 172.21.15.27:36780) 2024-09-21T01:22:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54e7e00640 1 -- 172.21.15.27:0/505772735 learned_addr learned my addr 172.21.15.27:0/505772735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54ed400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e810bf60 0x7f54e81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 -- 172.21.15.27:0/505772735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 msgr2=0x7f54e819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 -- 172.21.15.27:0/505772735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e810bf60 msgr2=0x7f54e81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e810bf60 0x7f54e81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 -- 172.21.15.27:0/505772735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e81a25d0 con 0x7f54e8105ce0 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54e7e00640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.231+0000 7f54eca00640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e8105ce0 0x7f54e819c6a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e9f0 tx=0x7f54dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.232+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00ccc0 con 0x7f54e8105ce0 2024-09-21T01:22:19.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.232+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e81a28c0 con 0x7f54e8105ce0 2024-09-21T01:22:19.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.232+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54dc0108e0 con 0x7f54e8105ce0 2024-09-21T01:22:19.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.232+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc018880 con 0x7f54e8105ce0 2024-09-21T01:22:19.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.232+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a9050 con 0x7f54e8105ce0 2024-09-21T01:22:19.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.233+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e81060f0 con 0x7f54e8105ce0 2024-09-21T01:22:19.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.236+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f54dc010430 con 0x7f54e8105ce0 2024-09-21T01:22:19.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.238+0000 7f54e5600640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f54d00b6680 0x7f54d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:19.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.238+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f54dc0e8270 con 0x7f54e8105ce0 2024-09-21T01:22:19.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.238+0000 7f54e7e00640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f54d00b6680 0x7f54d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:19.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.239+0000 7f54e7e00640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f54d00b6680 0x7f54d00b8b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f54e8103db0 tx=0x7f54d4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:19.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.239+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54dc0b27d0 con 0x7f54e8105ce0 2024-09-21T01:22:19.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.462+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f54e8063800 con 0x7f54d00b6680 2024-09-21T01:22:19.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.470+0000 7f54e5600640 1 -- 172.21.15.27:0/505772735 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f54e8063800 con 0x7f54d00b6680 2024-09-21T01:22:19.471 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-09-21T01:22:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f54d00b6680 msgr2=0x7f54d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f54d00b6680 0x7f54d00b8b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f54e8103db0 tx=0x7f54d4005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e8105ce0 msgr2=0x7f54e819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e8105ce0 0x7f54e819c6a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e9f0 tx=0x7f54dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 shutdown_connections 2024-09-21T01:22:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f54d00b6680 0x7f54d00b8b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f54e810bf60 0x7f54e81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54e810b610 0x7f54e819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 --2- 172.21.15.27:0/505772735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54e8105ce0 0x7f54e819c6a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 >> 172.21.15.27:0/505772735 conn(0x7f54e81016c0 msgr2=0x7f54e8109bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.473+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 shutdown_connections 2024-09-21T01:22:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:19.474+0000 7f54ef400640 1 -- 172.21.15.27:0/505772735 wait complete. 2024-09-21T01:22:19.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T01:22:19.559 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T01:22:19.570 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T01:22:19.584 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T01:22:19.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:22:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:19 smithi027 ceph-mon[30375]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:19 smithi027 ceph-mon[30375]: pgmap v397: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-21T01:22:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:19 smithi027 ceph-mon[30375]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T01:22:19.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:19 smithi027 ceph-mon[30375]: from='client.25493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:19.968 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:19.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:19 smithi089 ceph-mon[34486]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:20.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:19 smithi089 ceph-mon[34486]: pgmap v397: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-21T01:22:20.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:19 smithi089 ceph-mon[34486]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T01:22:20.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:19 smithi089 ceph-mon[34486]: from='client.25493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:19 smithi098 ceph-mon[34372]: from='client.25477 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:19 smithi098 ceph-mon[34372]: pgmap v397: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-21T01:22:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:19 smithi098 ceph-mon[34372]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T01:22:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:19 smithi098 ceph-mon[34372]: from='client.25493 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:20.732 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:20 smithi027 ceph-mon[30375]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 -- 172.21.15.27:0/1230168430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac114740 msgr2=0x7ff8ac116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:20.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 --2- 172.21.15.27:0/1230168430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac114740 0x7ff8ac116b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009ca0 tx=0x7ff8a402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 -- 172.21.15.27:0/1230168430 shutdown_connections 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 --2- 172.21.15.27:0/1230168430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac114740 0x7ff8ac116b60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 --2- 172.21.15.27:0/1230168430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac078290 0x7ff8ac114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 --2- 172.21.15.27:0/1230168430 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac077940 0x7ff8ac077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 -- 172.21.15.27:0/1230168430 >> 172.21.15.27:0/1230168430 conn(0x7ff8ac1017d0 msgr2=0x7ff8ac103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 -- 172.21.15.27:0/1230168430 shutdown_connections 2024-09-21T01:22:20.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.809+0000 7ff8b2200640 1 -- 172.21.15.27:0/1230168430 wait complete. 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.810+0000 7ff8b2200640 1 Processor -- start 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 -- start start 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 0x7ff8ac1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 0x7ff8ac1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac114740 0x7ff8ac1a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac11a100 con 0x7ff8ac077940 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac119f80 con 0x7ff8ac078290 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8b2200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac11a280 con 0x7ff8ac114740 2024-09-21T01:22:20.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 0x7ff8ac1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 0x7ff8ac1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac114740 0x7ff8ac1a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 0x7ff8ac1a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42980/0 (socket says 172.21.15.27:42980) 2024-09-21T01:22:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8a3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 0x7ff8ac1a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59772/0 (socket says 172.21.15.27:59772) 2024-09-21T01:22:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8a3e00640 1 -- 172.21.15.27:0/2466820610 learned_addr learned my addr 172.21.15.27:0/2466820610 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:20.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.811+0000 7ff8abe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac114740 0x7ff8ac1a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:36802/0 (socket says 172.21.15.27:36802) 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.812+0000 7ff8a3e00640 1 -- 172.21.15.27:0/2466820610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac114740 msgr2=0x7ff8ac1a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.812+0000 7ff8a3e00640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac114740 0x7ff8ac1a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.812+0000 7ff8a3e00640 1 -- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 msgr2=0x7ff8ac1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.812+0000 7ff8a3e00640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 0x7ff8ac1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.812+0000 7ff8a3e00640 1 -- 172.21.15.27:0/2466820610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac1a6a80 con 0x7ff8ac078290 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.812+0000 7ff8ab400640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 0x7ff8ac1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:20.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.813+0000 7ff8a3e00640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 0x7ff8ac1a10b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff89400d740 tx=0x7ff89400dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.813+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894017070 con 0x7ff8ac078290 2024-09-21T01:22:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.813+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff89400de50 con 0x7ff8ac078290 2024-09-21T01:22:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.813+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89400f620 con 0x7ff8ac078290 2024-09-21T01:22:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.813+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac1a6d70 con 0x7ff8ac078290 2024-09-21T01:22:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.814+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1ad4d0 con 0x7ff8ac078290 2024-09-21T01:22:20.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.815+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac077d50 con 0x7ff8ac078290 2024-09-21T01:22:20.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.817+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7ff89401a070 con 0x7ff8ac078290 2024-09-21T01:22:20.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.819+0000 7ff8a8c00640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff8900b6700 0x7ff8900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:20.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.819+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff8940e7260 con 0x7ff8ac078290 2024-09-21T01:22:20.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.819+0000 7ff8ab400640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff8900b6700 0x7ff8900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.820+0000 7ff8ab400640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff8900b6700 0x7ff8900b8c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fc0 tx=0x7ff89c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:20.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:20.821+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8940b18a0 con 0x7ff8ac078290 2024-09-21T01:22:21.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:20 smithi089 ceph-mon[34486]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:21.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.016+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8ac063800 con 0x7ff8900b6700 2024-09-21T01:22:21.024 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.023+0000 7ff8a8c00640 1 -- 172.21.15.27:0/2466820610 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340576 (secure 0 0 0) 0x7ff8ac063800 con 0x7ff8900b6700 2024-09-21T01:22:21.029 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:21.030 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:22:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.028+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff8900b6700 msgr2=0x7ff8900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.028+0000 7ff8b2200640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff8900b6700 0x7ff8900b8c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fc0 tx=0x7ff89c005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.028+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 msgr2=0x7ff8ac1a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.028+0000 7ff8b2200640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 0x7ff8ac1a10b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff89400d740 tx=0x7ff89400dc00 comp rx=0 tx=0).stop 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 shutdown_connections 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff8900b6700 0x7ff8900b8c10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8ac114740 0x7ff8ac1a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8ac078290 0x7ff8ac1a10b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 --2- 172.21.15.27:0/2466820610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8ac077940 0x7ff8ac1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 >> 172.21.15.27:0/2466820610 conn(0x7ff8ac1017d0 msgr2=0x7ff8ac116780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 shutdown_connections 2024-09-21T01:22:21.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:21.029+0000 7ff8b2200640 1 -- 172.21.15.27:0/2466820610 wait complete. 2024-09-21T01:22:21.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:20 smithi098 ceph-mon[34372]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:22:21.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:21 smithi027 ceph-mon[30375]: pgmap v398: 129 pgs: 129 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 103 KiB/s wr, 50 op/s 2024-09-21T01:22:21.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:21 smithi027 ceph-mon[30375]: from='client.25505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:22.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:21 smithi089 ceph-mon[34486]: pgmap v398: 129 pgs: 129 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 103 KiB/s wr, 50 op/s 2024-09-21T01:22:22.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:21 smithi089 ceph-mon[34486]: from='client.25505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:22.086 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-21T01:22:20.475400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1067132,"num_objects":103,"num_object_clones":0,"num_object_copies":311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":1724,"num_read_kb":1112,"num_write":546,"num_write_kb":3538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3019356,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":909,"ondisk_log_size":909,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372856,"kb_used_data":15228,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124568968,"statfs":{"total":1151940427776,"available":1151558623232,"internally_reserved":0,"allocated":15593472,"data_stored":9190585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":402583,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":442,"num_read_kb":346,"num_write":162,"num_write_kb":1233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004478"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:17.608226+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:22:17.608226+0000","last_peered":"2024-09-21T01:22:17.608226+0000","last_clean":"2024-09-21T01:22:17.608226+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:22:17.608226+0000","last_undegraded":"2024-09-21T01:22:17.608226+0000","last_fullsized":"2024-09-21T01:22:17.608226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824857+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:21:40.824857+0000","last_peered":"2024-09-21T01:21:40.824857+0000","last_clean":"2024-09-21T01:21:40.824857+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:21:40.824857+0000","last_undegraded":"2024-09-21T01:21:40.824857+0000","last_fullsized":"2024-09-21T01:21:40.824857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166128+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:20:42.166128+0000","last_peered":"2024-09-21T01:20:42.166128+0000","last_clean":"2024-09-21T01:20:42.166128+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:20:42.166128+0000","last_undegraded":"2024-09-21T01:20:42.166128+0000","last_fullsized":"2024-09-21T01:20:42.166128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213160+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:20:42.213160+0000","last_peered":"2024-09-21T01:20:42.213160+0000","last_clean":"2024-09-21T01:20:42.213160+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:20:42.213160+0000","last_undegraded":"2024-09-21T01:20:42.213160+0000","last_fullsized":"2024-09-21T01:20:42.213160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830390+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:21:40.830390+0000","last_peered":"2024-09-21T01:21:40.830390+0000","last_clean":"2024-09-21T01:21:40.830390+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:21:40.830390+0000","last_undegraded":"2024-09-21T01:21:40.830390+0000","last_fullsized":"2024-09-21T01:21:40.830390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168785+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:20:42.168785+0000","last_peered":"2024-09-21T01:20:42.168785+0000","last_clean":"2024-09-21T01:20:42.168785+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:20:42.168785+0000","last_undegraded":"2024-09-21T01:20:42.168785+0000","last_fullsized":"2024-09-21T01:20:42.168785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172136+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:20:42.172136+0000","last_peered":"2024-09-21T01:20:42.172136+0000","last_clean":"2024-09-21T01:20:42.172136+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:20:42.172136+0000","last_undegraded":"2024-09-21T01:20:42.172136+0000","last_fullsized":"2024-09-21T01:20:42.172136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837613+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:21:40.837613+0000","last_peered":"2024-09-21T01:21:40.837613+0000","last_clean":"2024-09-21T01:21:40.837613+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:21:40.837613+0000","last_undegraded":"2024-09-21T01:21:40.837613+0000","last_fullsized":"2024-09-21T01:21:40.837613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169405+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:20:42.169405+0000","last_peered":"2024-09-21T01:20:42.169405+0000","last_clean":"2024-09-21T01:20:42.169405+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:20:42.169405+0000","last_undegraded":"2024-09-21T01:20:42.169405+0000","last_fullsized":"2024-09-21T01:20:42.169405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172455+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:42.172455+0000","last_peered":"2024-09-21T01:20:42.172455+0000","last_clean":"2024-09-21T01:20:42.172455+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:42.172455+0000","last_undegraded":"2024-09-21T01:20:42.172455+0000","last_fullsized":"2024-09-21T01:20:42.172455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.840190+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:21:40.840190+0000","last_peered":"2024-09-21T01:21:40.840190+0000","last_clean":"2024-09-21T01:21:40.840190+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:21:40.840190+0000","last_undegraded":"2024-09-21T01:21:40.840190+0000","last_fullsized":"2024-09-21T01:21:40.840190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169337+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:20:42.169337+0000","last_peered":"2024-09-21T01:20:42.169337+0000","last_clean":"2024-09-21T01:20:42.169337+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:20:42.169337+0000","last_undegraded":"2024-09-21T01:20:42.169337+0000","last_fullsized":"2024-09-21T01:20:42.169337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837797+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:21:40.837797+0000","last_peered":"2024-09-21T01:21:40.837797+0000","last_clean":"2024-09-21T01:21:40.837797+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:21:40.837797+0000","last_undegraded":"2024-09-21T01:21:40.837797+0000","last_fullsized":"2024-09-21T01:21:40.837797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171800+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:42.171800+0000","last_peered":"2024-09-21T01:20:42.171800+0000","last_clean":"2024-09-21T01:20:42.171800+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:42.171800+0000","last_undegraded":"2024-09-21T01:20:42.171800+0000","last_fullsized":"2024-09-21T01:20:42.171800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170156+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:20:42.170156+0000","last_peered":"2024-09-21T01:20:42.170156+0000","last_clean":"2024-09-21T01:20:42.170156+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:20:42.170156+0000","last_undegraded":"2024-09-21T01:20:42.170156+0000","last_fullsized":"2024-09-21T01:20:42.170156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834742+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:21:40.834742+0000","last_peered":"2024-09-21T01:21:40.834742+0000","last_clean":"2024-09-21T01:21:40.834742+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:21:40.834742+0000","last_undegraded":"2024-09-21T01:21:40.834742+0000","last_fullsized":"2024-09-21T01:21:40.834742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166497+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:42.166497+0000","last_peered":"2024-09-21T01:20:42.166497+0000","last_clean":"2024-09-21T01:20:42.166497+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:42.166497+0000","last_undegraded":"2024-09-21T01:20:42.166497+0000","last_fullsized":"2024-09-21T01:20:42.166497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169996+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:20:42.169996+0000","last_peered":"2024-09-21T01:20:42.169996+0000","last_clean":"2024-09-21T01:20:42.169996+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:20:42.169996+0000","last_undegraded":"2024-09-21T01:20:42.169996+0000","last_fullsized":"2024-09-21T01:20:42.169996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.820540+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:21:40.820540+0000","last_peered":"2024-09-21T01:21:40.820540+0000","last_clean":"2024-09-21T01:21:40.820540+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:21:40.820540+0000","last_undegraded":"2024-09-21T01:21:40.820540+0000","last_fullsized":"2024-09-21T01:21:40.820540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212818+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:42.212818+0000","last_peered":"2024-09-21T01:20:42.212818+0000","last_clean":"2024-09-21T01:20:42.212818+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:42.212818+0000","last_undegraded":"2024-09-21T01:20:42.212818+0000","last_fullsized":"2024-09-21T01:20:42.212818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166998+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:20:42.166998+0000","last_peered":"2024-09-21T01:20:42.166998+0000","last_clean":"2024-09-21T01:20:42.166998+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:20:42.166998+0000","last_undegraded":"2024-09-21T01:20:42.166998+0000","last_fullsized":"2024-09-21T01:20:42.166998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.785150+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:21:40.785150+0000","last_peered":"2024-09-21T01:21:40.785150+0000","last_clean":"2024-09-21T01:21:40.785150+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:21:40.785150+0000","last_undegraded":"2024-09-21T01:21:40.785150+0000","last_fullsized":"2024-09-21T01:21:40.785150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173403+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:42.173403+0000","last_peered":"2024-09-21T01:20:42.173403+0000","last_clean":"2024-09-21T01:20:42.173403+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:42.173403+0000","last_undegraded":"2024-09-21T01:20:42.173403+0000","last_fullsized":"2024-09-21T01:20:42.173403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170133+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:20:42.170133+0000","last_peered":"2024-09-21T01:20:42.170133+0000","last_clean":"2024-09-21T01:20:42.170133+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.170133+0000","last_undegraded":"2024-09-21T01:20:42.170133+0000","last_fullsized":"2024-09-21T01:20:42.170133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834120+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:21:40.834120+0000","last_peered":"2024-09-21T01:21:40.834120+0000","last_clean":"2024-09-21T01:21:40.834120+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:21:40.834120+0000","last_undegraded":"2024-09-21T01:21:40.834120+0000","last_fullsized":"2024-09-21T01:21:40.834120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170465+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:42.170465+0000","last_peered":"2024-09-21T01:20:42.170465+0000","last_clean":"2024-09-21T01:20:42.170465+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:42.170465+0000","last_undegraded":"2024-09-21T01:20:42.170465+0000","last_fullsized":"2024-09-21T01:20:42.170465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166302+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:20:42.166302+0000","last_peered":"2024-09-21T01:20:42.166302+0000","last_clean":"2024-09-21T01:20:42.166302+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:20:42.166302+0000","last_undegraded":"2024-09-21T01:20:42.166302+0000","last_fullsized":"2024-09-21T01:20:42.166302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834414+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:21:40.834414+0000","last_peered":"2024-09-21T01:21:40.834414+0000","last_clean":"2024-09-21T01:21:40.834414+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:21:40.834414+0000","last_undegraded":"2024-09-21T01:21:40.834414+0000","last_fullsized":"2024-09-21T01:21:40.834414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'101","reported_seq":232,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.973077+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:22:18.973077+0000","last_peered":"2024-09-21T01:22:18.973077+0000","last_clean":"2024-09-21T01:22:18.973077+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:22:18.973077+0000","last_undegraded":"2024-09-21T01:22:18.973077+0000","last_fullsized":"2024-09-21T01:22:18.973077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":184,"num_write_kb":1223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212484+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:20:42.212484+0000","last_peered":"2024-09-21T01:20:42.212484+0000","last_clean":"2024-09-21T01:20:42.212484+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.212484+0000","last_undegraded":"2024-09-21T01:20:42.212484+0000","last_fullsized":"2024-09-21T01:20:42.212484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.814422+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:21:40.814422+0000","last_peered":"2024-09-21T01:21:40.814422+0000","last_clean":"2024-09-21T01:21:40.814422+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:21:40.814422+0000","last_undegraded":"2024-09-21T01:21:40.814422+0000","last_fullsized":"2024-09-21T01:21:40.814422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.185170+0000","last_change":"2024-09-21T01:20:42.184446+0000","last_active":"2024-09-21T01:20:42.185170+0000","last_peered":"2024-09-21T01:20:42.185170+0000","last_clean":"2024-09-21T01:20:42.185170+0000","last_became_active":"2024-09-21T01:20:42.172338+0000","last_became_peered":"2024-09-21T01:20:42.172338+0000","last_unstale":"2024-09-21T01:20:42.185170+0000","last_undegraded":"2024-09-21T01:20:42.185170+0000","last_fullsized":"2024-09-21T01:20:42.185170+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:12.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168852+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:20:42.168852+0000","last_peered":"2024-09-21T01:20:42.168852+0000","last_clean":"2024-09-21T01:20:42.168852+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:20:42.168852+0000","last_undegraded":"2024-09-21T01:20:42.168852+0000","last_fullsized":"2024-09-21T01:20:42.168852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832976+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:21:40.832976+0000","last_peered":"2024-09-21T01:21:40.832976+0000","last_clean":"2024-09-21T01:21:40.832976+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:21:40.832976+0000","last_undegraded":"2024-09-21T01:21:40.832976+0000","last_fullsized":"2024-09-21T01:21:40.832976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167193+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:20:42.167193+0000","last_peered":"2024-09-21T01:20:42.167193+0000","last_clean":"2024-09-21T01:20:42.167193+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:20:42.167193+0000","last_undegraded":"2024-09-21T01:20:42.167193+0000","last_fullsized":"2024-09-21T01:20:42.167193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171244+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:20:42.171244+0000","last_peered":"2024-09-21T01:20:42.171244+0000","last_clean":"2024-09-21T01:20:42.171244+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:20:42.171244+0000","last_undegraded":"2024-09-21T01:20:42.171244+0000","last_fullsized":"2024-09-21T01:20:42.171244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829906+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:21:40.829906+0000","last_peered":"2024-09-21T01:21:40.829906+0000","last_clean":"2024-09-21T01:21:40.829906+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:21:40.829906+0000","last_undegraded":"2024-09-21T01:21:40.829906+0000","last_fullsized":"2024-09-21T01:21:40.829906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":188,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.611456+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:22:18.611456+0000","last_peered":"2024-09-21T01:22:18.611456+0000","last_clean":"2024-09-21T01:22:18.611456+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:22:18.611456+0000","last_undegraded":"2024-09-21T01:22:18.611456+0000","last_fullsized":"2024-09-21T01:22:18.611456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368522+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:20:42.368522+0000","last_peered":"2024-09-21T01:20:42.368522+0000","last_clean":"2024-09-21T01:20:42.368522+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:20:42.368522+0000","last_undegraded":"2024-09-21T01:20:42.368522+0000","last_fullsized":"2024-09-21T01:20:42.368522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824000+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:21:40.824000+0000","last_peered":"2024-09-21T01:21:40.824000+0000","last_clean":"2024-09-21T01:21:40.824000+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:21:40.824000+0000","last_undegraded":"2024-09-21T01:21:40.824000+0000","last_fullsized":"2024-09-21T01:21:40.824000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":187,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:17.608243+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:22:17.608243+0000","last_peered":"2024-09-21T01:22:17.608243+0000","last_clean":"2024-09-21T01:22:17.608243+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:22:17.608243+0000","last_undegraded":"2024-09-21T01:22:17.608243+0000","last_fullsized":"2024-09-21T01:22:17.608243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368663+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:20:42.368663+0000","last_peered":"2024-09-21T01:20:42.368663+0000","last_clean":"2024-09-21T01:20:42.368663+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:20:42.368663+0000","last_undegraded":"2024-09-21T01:20:42.368663+0000","last_fullsized":"2024-09-21T01:20:42.368663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.811679+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:21:40.811679+0000","last_peered":"2024-09-21T01:21:40.811679+0000","last_clean":"2024-09-21T01:21:40.811679+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:21:40.811679+0000","last_undegraded":"2024-09-21T01:21:40.811679+0000","last_fullsized":"2024-09-21T01:21:40.811679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166486+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:42.166486+0000","last_peered":"2024-09-21T01:20:42.166486+0000","last_clean":"2024-09-21T01:20:42.166486+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:42.166486+0000","last_undegraded":"2024-09-21T01:20:42.166486+0000","last_fullsized":"2024-09-21T01:20:42.166486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171878+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:20:42.171878+0000","last_peered":"2024-09-21T01:20:42.171878+0000","last_clean":"2024-09-21T01:20:42.171878+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:20:42.171878+0000","last_undegraded":"2024-09-21T01:20:42.171878+0000","last_fullsized":"2024-09-21T01:20:42.171878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837018+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:21:40.837018+0000","last_peered":"2024-09-21T01:21:40.837018+0000","last_clean":"2024-09-21T01:21:40.837018+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:21:40.837018+0000","last_undegraded":"2024-09-21T01:21:40.837018+0000","last_fullsized":"2024-09-21T01:21:40.837018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169791+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:42.169791+0000","last_peered":"2024-09-21T01:20:42.169791+0000","last_clean":"2024-09-21T01:20:42.169791+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:42.169791+0000","last_undegraded":"2024-09-21T01:20:42.169791+0000","last_fullsized":"2024-09-21T01:20:42.169791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166664+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:20:42.166664+0000","last_peered":"2024-09-21T01:20:42.166664+0000","last_clean":"2024-09-21T01:20:42.166664+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:20:42.166664+0000","last_undegraded":"2024-09-21T01:20:42.166664+0000","last_fullsized":"2024-09-21T01:20:42.166664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166645+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:20:42.166645+0000","last_peered":"2024-09-21T01:20:42.166645+0000","last_clean":"2024-09-21T01:20:42.166645+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:20:42.166645+0000","last_undegraded":"2024-09-21T01:20:42.166645+0000","last_fullsized":"2024-09-21T01:20:42.166645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838028+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:21:40.838028+0000","last_peered":"2024-09-21T01:21:40.838028+0000","last_clean":"2024-09-21T01:21:40.838028+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:21:40.838028+0000","last_undegraded":"2024-09-21T01:21:40.838028+0000","last_fullsized":"2024-09-21T01:21:40.838028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.348297+0000","last_change":"2024-09-21T01:20:42.188731+0000","last_active":"2024-09-21T01:22:18.348297+0000","last_peered":"2024-09-21T01:22:18.348297+0000","last_clean":"2024-09-21T01:22:18.348297+0000","last_became_active":"2024-09-21T01:20:42.172041+0000","last_became_peered":"2024-09-21T01:20:42.172041+0000","last_unstale":"2024-09-21T01:22:18.348297+0000","last_undegraded":"2024-09-21T01:22:18.348297+0000","last_fullsized":"2024-09-21T01:22:18.348297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:42.392074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173149+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:20:42.173149+0000","last_peered":"2024-09-21T01:20:42.173149+0000","last_clean":"2024-09-21T01:20:42.173149+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:20:42.173149+0000","last_undegraded":"2024-09-21T01:20:42.173149+0000","last_fullsized":"2024-09-21T01:20:42.173149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831868+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:21:40.831868+0000","last_peered":"2024-09-21T01:21:40.831868+0000","last_clean":"2024-09-21T01:21:40.831868+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:21:40.831868+0000","last_undegraded":"2024-09-21T01:21:40.831868+0000","last_fullsized":"2024-09-21T01:21:40.831868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174409+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:20:42.174409+0000","last_peered":"2024-09-21T01:20:42.174409+0000","last_clean":"2024-09-21T01:20:42.174409+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:20:42.174409+0000","last_undegraded":"2024-09-21T01:20:42.174409+0000","last_fullsized":"2024-09-21T01:20:42.174409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171335+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:20:42.171335+0000","last_peered":"2024-09-21T01:20:42.171335+0000","last_clean":"2024-09-21T01:20:42.171335+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:20:42.171335+0000","last_undegraded":"2024-09-21T01:20:42.171335+0000","last_fullsized":"2024-09-21T01:20:42.171335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830380+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:21:40.830380+0000","last_peered":"2024-09-21T01:21:40.830380+0000","last_clean":"2024-09-21T01:21:40.830380+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:21:40.830380+0000","last_undegraded":"2024-09-21T01:21:40.830380+0000","last_fullsized":"2024-09-21T01:21:40.830380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368686+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:42.368686+0000","last_peered":"2024-09-21T01:20:42.368686+0000","last_clean":"2024-09-21T01:20:42.368686+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:42.368686+0000","last_undegraded":"2024-09-21T01:20:42.368686+0000","last_fullsized":"2024-09-21T01:20:42.368686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173156+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:20:42.173156+0000","last_peered":"2024-09-21T01:20:42.173156+0000","last_clean":"2024-09-21T01:20:42.173156+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:20:42.173156+0000","last_undegraded":"2024-09-21T01:20:42.173156+0000","last_fullsized":"2024-09-21T01:20:42.173156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824620+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:21:40.824620+0000","last_peered":"2024-09-21T01:21:40.824620+0000","last_clean":"2024-09-21T01:21:40.824620+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:21:40.824620+0000","last_undegraded":"2024-09-21T01:21:40.824620+0000","last_fullsized":"2024-09-21T01:21:40.824620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.189333+0000","last_change":"2024-09-21T01:20:42.188808+0000","last_active":"2024-09-21T01:20:42.189333+0000","last_peered":"2024-09-21T01:20:42.189333+0000","last_clean":"2024-09-21T01:20:42.189333+0000","last_became_active":"2024-09-21T01:20:42.175348+0000","last_became_peered":"2024-09-21T01:20:42.175348+0000","last_unstale":"2024-09-21T01:20:42.189333+0000","last_undegraded":"2024-09-21T01:20:42.189333+0000","last_fullsized":"2024-09-21T01:20:42.189333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.746569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173579+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:20:42.173579+0000","last_peered":"2024-09-21T01:20:42.173579+0000","last_clean":"2024-09-21T01:20:42.173579+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:20:42.173579+0000","last_undegraded":"2024-09-21T01:20:42.173579+0000","last_fullsized":"2024-09-21T01:20:42.173579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.839227+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:21:40.839227+0000","last_peered":"2024-09-21T01:21:40.839227+0000","last_clean":"2024-09-21T01:21:40.839227+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:21:40.839227+0000","last_undegraded":"2024-09-21T01:21:40.839227+0000","last_fullsized":"2024-09-21T01:21:40.839227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171236+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:42.171236+0000","last_peered":"2024-09-21T01:20:42.171236+0000","last_clean":"2024-09-21T01:20:42.171236+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:42.171236+0000","last_undegraded":"2024-09-21T01:20:42.171236+0000","last_fullsized":"2024-09-21T01:20:42.171236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368646+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:20:42.368646+0000","last_peered":"2024-09-21T01:20:42.368646+0000","last_clean":"2024-09-21T01:20:42.368646+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:20:42.368646+0000","last_undegraded":"2024-09-21T01:20:42.368646+0000","last_fullsized":"2024-09-21T01:20:42.368646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831667+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:21:40.831667+0000","last_peered":"2024-09-21T01:21:40.831667+0000","last_clean":"2024-09-21T01:21:40.831667+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:21:40.831667+0000","last_undegraded":"2024-09-21T01:21:40.831667+0000","last_fullsized":"2024-09-21T01:21:40.831667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":191,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.610592+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:22:18.610592+0000","last_peered":"2024-09-21T01:22:18.610592+0000","last_clean":"2024-09-21T01:22:18.610592+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:22:18.610592+0000","last_undegraded":"2024-09-21T01:22:18.610592+0000","last_fullsized":"2024-09-21T01:22:18.610592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172600+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:20:42.172600+0000","last_peered":"2024-09-21T01:20:42.172600+0000","last_clean":"2024-09-21T01:20:42.172600+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:20:42.172600+0000","last_undegraded":"2024-09-21T01:20:42.172600+0000","last_fullsized":"2024-09-21T01:20:42.172600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832208+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:21:40.832208+0000","last_peered":"2024-09-21T01:21:40.832208+0000","last_clean":"2024-09-21T01:21:40.832208+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:21:40.832208+0000","last_undegraded":"2024-09-21T01:21:40.832208+0000","last_fullsized":"2024-09-21T01:21:40.832208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173980+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:42.173980+0000","last_peered":"2024-09-21T01:20:42.173980+0000","last_clean":"2024-09-21T01:20:42.173980+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:42.173980+0000","last_undegraded":"2024-09-21T01:20:42.173980+0000","last_fullsized":"2024-09-21T01:20:42.173980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168815+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:20:42.168815+0000","last_peered":"2024-09-21T01:20:42.168815+0000","last_clean":"2024-09-21T01:20:42.168815+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:20:42.168815+0000","last_undegraded":"2024-09-21T01:20:42.168815+0000","last_fullsized":"2024-09-21T01:20:42.168815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829263+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:21:40.829263+0000","last_peered":"2024-09-21T01:21:40.829263+0000","last_clean":"2024-09-21T01:21:40.829263+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:21:40.829263+0000","last_undegraded":"2024-09-21T01:21:40.829263+0000","last_fullsized":"2024-09-21T01:21:40.829263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166698+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:42.166698+0000","last_peered":"2024-09-21T01:20:42.166698+0000","last_clean":"2024-09-21T01:20:42.166698+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:42.166698+0000","last_undegraded":"2024-09-21T01:20:42.166698+0000","last_fullsized":"2024-09-21T01:20:42.166698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173990+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:20:42.173990+0000","last_peered":"2024-09-21T01:20:42.173990+0000","last_clean":"2024-09-21T01:20:42.173990+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:20:42.173990+0000","last_undegraded":"2024-09-21T01:20:42.173990+0000","last_fullsized":"2024-09-21T01:20:42.173990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.810339+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:21:40.810339+0000","last_peered":"2024-09-21T01:21:40.810339+0000","last_clean":"2024-09-21T01:21:40.810339+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:21:40.810339+0000","last_undegraded":"2024-09-21T01:21:40.810339+0000","last_fullsized":"2024-09-21T01:21:40.810339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":392,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:20.333237+0000","last_change":"2024-09-21T01:20:42.182111+0000","last_active":"2024-09-21T01:22:20.333237+0000","last_peered":"2024-09-21T01:22:20.333237+0000","last_clean":"2024-09-21T01:22:20.333237+0000","last_became_active":"2024-09-21T01:20:42.174484+0000","last_became_peered":"2024-09-21T01:20:42.174484+0000","last_unstale":"2024-09-21T01:22:20.333237+0000","last_undegraded":"2024-09-21T01:22:20.333237+0000","last_fullsized":"2024-09-21T01:22:20.333237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:45.448208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":281,"num_read_kb":214,"num_write":44,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":66212,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166872+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:20:42.166872+0000","last_peered":"2024-09-21T01:20:42.166872+0000","last_clean":"2024-09-21T01:20:42.166872+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:20:42.166872+0000","last_undegraded":"2024-09-21T01:20:42.166872+0000","last_fullsized":"2024-09-21T01:20:42.166872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838943+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:21:40.838943+0000","last_peered":"2024-09-21T01:21:40.838943+0000","last_clean":"2024-09-21T01:21:40.838943+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:21:40.838943+0000","last_undegraded":"2024-09-21T01:21:40.838943+0000","last_fullsized":"2024-09-21T01:21:40.838943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174291+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:42.174291+0000","last_peered":"2024-09-21T01:20:42.174291+0000","last_clean":"2024-09-21T01:20:42.174291+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:42.174291+0000","last_undegraded":"2024-09-21T01:20:42.174291+0000","last_fullsized":"2024-09-21T01:20:42.174291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169605+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:20:42.169605+0000","last_peered":"2024-09-21T01:20:42.169605+0000","last_clean":"2024-09-21T01:20:42.169605+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:20:42.169605+0000","last_undegraded":"2024-09-21T01:20:42.169605+0000","last_fullsized":"2024-09-21T01:20:42.169605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.822722+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:21:40.822722+0000","last_peered":"2024-09-21T01:21:40.822722+0000","last_clean":"2024-09-21T01:21:40.822722+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:21:40.822722+0000","last_undegraded":"2024-09-21T01:21:40.822722+0000","last_fullsized":"2024-09-21T01:21:40.822722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167148+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:42.167148+0000","last_peered":"2024-09-21T01:20:42.167148+0000","last_clean":"2024-09-21T01:20:42.167148+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:42.167148+0000","last_undegraded":"2024-09-21T01:20:42.167148+0000","last_fullsized":"2024-09-21T01:20:42.167148+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166594+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:20:42.166594+0000","last_peered":"2024-09-21T01:20:42.166594+0000","last_clean":"2024-09-21T01:20:42.166594+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:20:42.166594+0000","last_undegraded":"2024-09-21T01:20:42.166594+0000","last_fullsized":"2024-09-21T01:20:42.166594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837693+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:21:40.837693+0000","last_peered":"2024-09-21T01:21:40.837693+0000","last_clean":"2024-09-21T01:21:40.837693+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:21:40.837693+0000","last_undegraded":"2024-09-21T01:21:40.837693+0000","last_fullsized":"2024-09-21T01:21:40.837693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":322,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:19.612424+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:22:19.612424+0000","last_peered":"2024-09-21T01:22:19.612424+0000","last_clean":"2024-09-21T01:22:19.612424+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:22:19.612424+0000","last_undegraded":"2024-09-21T01:22:19.612424+0000","last_fullsized":"2024-09-21T01:22:19.612424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212589+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:20:42.212589+0000","last_peered":"2024-09-21T01:20:42.212589+0000","last_clean":"2024-09-21T01:20:42.212589+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:20:42.212589+0000","last_undegraded":"2024-09-21T01:20:42.212589+0000","last_fullsized":"2024-09-21T01:20:42.212589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171758+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:20:42.171758+0000","last_peered":"2024-09-21T01:20:42.171758+0000","last_clean":"2024-09-21T01:20:42.171758+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:20:42.171758+0000","last_undegraded":"2024-09-21T01:20:42.171758+0000","last_fullsized":"2024-09-21T01:20:42.171758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.827678+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:21:40.827678+0000","last_peered":"2024-09-21T01:21:40.827678+0000","last_clean":"2024-09-21T01:21:40.827678+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:21:40.827678+0000","last_undegraded":"2024-09-21T01:21:40.827678+0000","last_fullsized":"2024-09-21T01:21:40.827678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173442+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:42.173442+0000","last_peered":"2024-09-21T01:20:42.173442+0000","last_clean":"2024-09-21T01:20:42.173442+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:42.173442+0000","last_undegraded":"2024-09-21T01:20:42.173442+0000","last_fullsized":"2024-09-21T01:20:42.173442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169158+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:20:42.169158+0000","last_peered":"2024-09-21T01:20:42.169158+0000","last_clean":"2024-09-21T01:20:42.169158+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:20:42.169158+0000","last_undegraded":"2024-09-21T01:20:42.169158+0000","last_fullsized":"2024-09-21T01:20:42.169158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833231+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:21:40.833231+0000","last_peered":"2024-09-21T01:21:40.833231+0000","last_clean":"2024-09-21T01:21:40.833231+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:21:40.833231+0000","last_undegraded":"2024-09-21T01:21:40.833231+0000","last_fullsized":"2024-09-21T01:21:40.833231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368185+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:42.368185+0000","last_peered":"2024-09-21T01:20:42.368185+0000","last_clean":"2024-09-21T01:20:42.368185+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:42.368185+0000","last_undegraded":"2024-09-21T01:20:42.368185+0000","last_fullsized":"2024-09-21T01:20:42.368185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169957+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:42.169957+0000","last_peered":"2024-09-21T01:20:42.169957+0000","last_clean":"2024-09-21T01:20:42.169957+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:42.169957+0000","last_undegraded":"2024-09-21T01:20:42.169957+0000","last_fullsized":"2024-09-21T01:20:42.169957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833060+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:21:40.833060+0000","last_peered":"2024-09-21T01:21:40.833060+0000","last_clean":"2024-09-21T01:21:40.833060+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:21:40.833060+0000","last_undegraded":"2024-09-21T01:21:40.833060+0000","last_fullsized":"2024-09-21T01:21:40.833060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166300+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:20:42.166300+0000","last_peered":"2024-09-21T01:20:42.166300+0000","last_clean":"2024-09-21T01:20:42.166300+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:20:42.166300+0000","last_undegraded":"2024-09-21T01:20:42.166300+0000","last_fullsized":"2024-09-21T01:20:42.166300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":190,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:15.607513+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:22:15.607513+0000","last_peered":"2024-09-21T01:22:15.607513+0000","last_clean":"2024-09-21T01:22:15.607513+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:22:15.607513+0000","last_undegraded":"2024-09-21T01:22:15.607513+0000","last_fullsized":"2024-09-21T01:22:15.607513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1465,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.812341+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:21:40.812341+0000","last_peered":"2024-09-21T01:21:40.812341+0000","last_clean":"2024-09-21T01:21:40.812341+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:21:40.812341+0000","last_undegraded":"2024-09-21T01:21:40.812341+0000","last_fullsized":"2024-09-21T01:21:40.812341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166566+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:20:42.166566+0000","last_peered":"2024-09-21T01:20:42.166566+0000","last_clean":"2024-09-21T01:20:42.166566+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:20:42.166566+0000","last_undegraded":"2024-09-21T01:20:42.166566+0000","last_fullsized":"2024-09-21T01:20:42.166566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.825083+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:21:40.825083+0000","last_peered":"2024-09-21T01:21:40.825083+0000","last_clean":"2024-09-21T01:21:40.825083+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:21:40.825083+0000","last_undegraded":"2024-09-21T01:21:40.825083+0000","last_fullsized":"2024-09-21T01:21:40.825083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.823972+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:21:40.823972+0000","last_peered":"2024-09-21T01:21:40.823972+0000","last_clean":"2024-09-21T01:21:40.823972+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:21:40.823972+0000","last_undegraded":"2024-09-21T01:21:40.823972+0000","last_fullsized":"2024-09-21T01:21:40.823972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831722+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:21:40.831722+0000","last_peered":"2024-09-21T01:21:40.831722+0000","last_clean":"2024-09-21T01:21:40.831722+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:21:40.831722+0000","last_undegraded":"2024-09-21T01:21:40.831722+0000","last_fullsized":"2024-09-21T01:21:40.831722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830027+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:21:40.830027+0000","last_peered":"2024-09-21T01:21:40.830027+0000","last_clean":"2024-09-21T01:21:40.830027+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:21:40.830027+0000","last_undegraded":"2024-09-21T01:21:40.830027+0000","last_fullsized":"2024-09-21T01:21:40.830027+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829116+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:21:40.829116+0000","last_peered":"2024-09-21T01:21:40.829116+0000","last_clean":"2024-09-21T01:21:40.829116+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:21:40.829116+0000","last_undegraded":"2024-09-21T01:21:40.829116+0000","last_fullsized":"2024-09-21T01:21:40.829116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824234+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:21:40.824234+0000","last_peered":"2024-09-21T01:21:40.824234+0000","last_clean":"2024-09-21T01:21:40.824234+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:21:40.824234+0000","last_undegraded":"2024-09-21T01:21:40.824234+0000","last_fullsized":"2024-09-21T01:21:40.824234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831569+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:21:40.831569+0000","last_peered":"2024-09-21T01:21:40.831569+0000","last_clean":"2024-09-21T01:21:40.831569+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:21:40.831569+0000","last_undegraded":"2024-09-21T01:21:40.831569+0000","last_fullsized":"2024-09-21T01:21:40.831569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832623+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:21:40.832623+0000","last_peered":"2024-09-21T01:21:40.832623+0000","last_clean":"2024-09-21T01:21:40.832623+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:21:40.832623+0000","last_undegraded":"2024-09-21T01:21:40.832623+0000","last_fullsized":"2024-09-21T01:21:40.832623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832822+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:21:40.832822+0000","last_peered":"2024-09-21T01:21:40.832822+0000","last_clean":"2024-09-21T01:21:40.832822+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:21:40.832822+0000","last_undegraded":"2024-09-21T01:21:40.832822+0000","last_fullsized":"2024-09-21T01:21:40.832822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833468+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:21:40.833468+0000","last_peered":"2024-09-21T01:21:40.833468+0000","last_clean":"2024-09-21T01:21:40.833468+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:21:40.833468+0000","last_undegraded":"2024-09-21T01:21:40.833468+0000","last_fullsized":"2024-09-21T01:21:40.833468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838351+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:21:40.838351+0000","last_peered":"2024-09-21T01:21:40.838351+0000","last_clean":"2024-09-21T01:21:40.838351+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:21:40.838351+0000","last_undegraded":"2024-09-21T01:21:40.838351+0000","last_fullsized":"2024-09-21T01:21:40.838351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830634+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:21:40.830634+0000","last_peered":"2024-09-21T01:21:40.830634+0000","last_clean":"2024-09-21T01:21:40.830634+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:21:40.830634+0000","last_undegraded":"2024-09-21T01:21:40.830634+0000","last_fullsized":"2024-09-21T01:21:40.830634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.823049+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:21:40.823049+0000","last_peered":"2024-09-21T01:21:40.823049+0000","last_clean":"2024-09-21T01:21:40.823049+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:21:40.823049+0000","last_undegraded":"2024-09-21T01:21:40.823049+0000","last_fullsized":"2024-09-21T01:21:40.823049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834378+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:21:40.834378+0000","last_peered":"2024-09-21T01:21:40.834378+0000","last_clean":"2024-09-21T01:21:40.834378+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:21:40.834378+0000","last_undegraded":"2024-09-21T01:21:40.834378+0000","last_fullsized":"2024-09-21T01:21:40.834378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837211+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:21:40.837211+0000","last_peered":"2024-09-21T01:21:40.837211+0000","last_clean":"2024-09-21T01:21:40.837211+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:21:40.837211+0000","last_undegraded":"2024-09-21T01:21:40.837211+0000","last_fullsized":"2024-09-21T01:21:40.837211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830574+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:21:40.830574+0000","last_peered":"2024-09-21T01:21:40.830574+0000","last_clean":"2024-09-21T01:21:40.830574+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:21:40.830574+0000","last_undegraded":"2024-09-21T01:21:40.830574+0000","last_fullsized":"2024-09-21T01:21:40.830574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.835525+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:21:40.835525+0000","last_peered":"2024-09-21T01:21:40.835525+0000","last_clean":"2024-09-21T01:21:40.835525+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:21:40.835525+0000","last_undegraded":"2024-09-21T01:21:40.835525+0000","last_fullsized":"2024-09-21T01:21:40.835525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.828196+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:21:40.828196+0000","last_peered":"2024-09-21T01:21:40.828196+0000","last_clean":"2024-09-21T01:21:40.828196+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:21:40.828196+0000","last_undegraded":"2024-09-21T01:21:40.828196+0000","last_fullsized":"2024-09-21T01:21:40.828196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.827534+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:21:40.827534+0000","last_peered":"2024-09-21T01:21:40.827534+0000","last_clean":"2024-09-21T01:21:40.827534+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:21:40.827534+0000","last_undegraded":"2024-09-21T01:21:40.827534+0000","last_fullsized":"2024-09-21T01:21:40.827534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829445+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:21:40.829445+0000","last_peered":"2024-09-21T01:21:40.829445+0000","last_clean":"2024-09-21T01:21:40.829445+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:21:40.829445+0000","last_undegraded":"2024-09-21T01:21:40.829445+0000","last_fullsized":"2024-09-21T01:21:40.829445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.810535+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:21:40.810535+0000","last_peered":"2024-09-21T01:21:40.810535+0000","last_clean":"2024-09-21T01:21:40.810535+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:21:40.810535+0000","last_undegraded":"2024-09-21T01:21:40.810535+0000","last_fullsized":"2024-09-21T01:21:40.810535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838099+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:21:40.838099+0000","last_peered":"2024-09-21T01:21:40.838099+0000","last_clean":"2024-09-21T01:21:40.838099+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:21:40.838099+0000","last_undegraded":"2024-09-21T01:21:40.838099+0000","last_fullsized":"2024-09-21T01:21:40.838099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.825971+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:21:40.825971+0000","last_peered":"2024-09-21T01:21:40.825971+0000","last_clean":"2024-09-21T01:21:40.825971+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:21:40.825971+0000","last_undegraded":"2024-09-21T01:21:40.825971+0000","last_fullsized":"2024-09-21T01:21:40.825971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837719+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:21:40.837719+0000","last_peered":"2024-09-21T01:21:40.837719+0000","last_clean":"2024-09-21T01:21:40.837719+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:21:40.837719+0000","last_undegraded":"2024-09-21T01:21:40.837719+0000","last_fullsized":"2024-09-21T01:21:40.837719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834644+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:21:40.834644+0000","last_peered":"2024-09-21T01:21:40.834644+0000","last_clean":"2024-09-21T01:21:40.834644+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:21:40.834644+0000","last_undegraded":"2024-09-21T01:21:40.834644+0000","last_fullsized":"2024-09-21T01:21:40.834644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830668+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:21:40.830668+0000","last_peered":"2024-09-21T01:21:40.830668+0000","last_clean":"2024-09-21T01:21:40.830668+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:21:40.830668+0000","last_undegraded":"2024-09-21T01:21:40.830668+0000","last_fullsized":"2024-09-21T01:21:40.830668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.826190+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:21:40.826190+0000","last_peered":"2024-09-21T01:21:40.826190+0000","last_clean":"2024-09-21T01:21:40.826190+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:21:40.826190+0000","last_undegraded":"2024-09-21T01:21:40.826190+0000","last_fullsized":"2024-09-21T01:21:40.826190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.821398+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:21:40.821398+0000","last_peered":"2024-09-21T01:21:40.821398+0000","last_clean":"2024-09-21T01:21:40.821398+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:21:40.821398+0000","last_undegraded":"2024-09-21T01:21:40.821398+0000","last_fullsized":"2024-09-21T01:21:40.821398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.817572+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:21:40.817572+0000","last_peered":"2024-09-21T01:21:40.817572+0000","last_clean":"2024-09-21T01:21:40.817572+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:21:40.817572+0000","last_undegraded":"2024-09-21T01:21:40.817572+0000","last_fullsized":"2024-09-21T01:21:40.817572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830525+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:21:40.830525+0000","last_peered":"2024-09-21T01:21:40.830525+0000","last_clean":"2024-09-21T01:21:40.830525+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:21:40.830525+0000","last_undegraded":"2024-09-21T01:21:40.830525+0000","last_fullsized":"2024-09-21T01:21:40.830525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.803325+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:21:40.803325+0000","last_peered":"2024-09-21T01:21:40.803325+0000","last_clean":"2024-09-21T01:21:40.803325+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:21:40.803325+0000","last_undegraded":"2024-09-21T01:21:40.803325+0000","last_fullsized":"2024-09-21T01:21:40.803325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830302+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:21:40.830302+0000","last_peered":"2024-09-21T01:21:40.830302+0000","last_clean":"2024-09-21T01:21:40.830302+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:21:40.830302+0000","last_undegraded":"2024-09-21T01:21:40.830302+0000","last_fullsized":"2024-09-21T01:21:40.830302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":476530,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1643,"num_read_kb":1043,"num_write":421,"num_write_kb":1386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":67516,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1396736,"data_stored":1028189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":535509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874299,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29192,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715960,"statfs":{"total":95995035648,"available":95965143040,"internally_reserved":0,"allocated":2158592,"data_stored":1595721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037824,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1466368,"data_stored":928521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234053,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":535549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430284,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1133555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593810,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790040,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1128214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":609191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149794,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":530517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378728,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":531732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":598611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705780,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":532265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":77528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":471170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:22:22.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:22:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:21 smithi098 ceph-mon[34372]: pgmap v398: 129 pgs: 129 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 103 KiB/s wr, 50 op/s 2024-09-21T01:22:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:21 smithi098 ceph-mon[34372]: from='client.25505 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 -- 172.21.15.27:0/832569193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108077940 msgr2=0x7fe108077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 --2- 172.21.15.27:0/832569193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108077940 0x7fe108077d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009900 tx=0x7fe0f802a410 comp rx=0 tx=0).stop 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 -- 172.21.15.27:0/832569193 shutdown_connections 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 --2- 172.21.15.27:0/832569193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108114740 0x7fe108116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 --2- 172.21.15.27:0/832569193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe108078290 0x7fe108114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 --2- 172.21.15.27:0/832569193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108077940 0x7fe108077d50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 -- 172.21.15.27:0/832569193 >> 172.21.15.27:0/832569193 conn(0x7fe1081017d0 msgr2=0x7fe108103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:23.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 -- 172.21.15.27:0/832569193 shutdown_connections 2024-09-21T01:22:23.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.232+0000 7fe10d200640 1 -- 172.21.15.27:0/832569193 wait complete. 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.233+0000 7fe10d200640 1 Processor -- start 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.233+0000 7fe10d200640 1 -- start start 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.233+0000 7fe10d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108077940 0x7fe1081a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe10d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 0x7fe1081a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe10d200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe108114740 0x7fe1081a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe10d200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe10811a120 con 0x7fe108078290 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe10d200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe108119fa0 con 0x7fe108077940 2024-09-21T01:22:23.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe10d200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe10811a2a0 con 0x7fe108114740 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 0x7fe1081a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108077940 0x7fe1081a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 0x7fe1081a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42992/0 (socket says 172.21.15.27:42992) 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 -- 172.21.15.27:0/2227341156 learned_addr learned my addr 172.21.15.27:0/2227341156 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe106800640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe108114740 0x7fe1081a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 -- 172.21.15.27:0/2227341156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe108114740 msgr2=0x7fe1081a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe108114740 0x7fe1081a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 -- 172.21.15.27:0/2227341156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108077940 msgr2=0x7fe1081a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108077940 0x7fe1081a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105400640 1 -- 172.21.15.27:0/2227341156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1081a6a60 con 0x7fe108078290 2024-09-21T01:22:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.234+0000 7fe105e00640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108077940 0x7fe1081a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.235+0000 7fe105400640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 0x7fe1081a1090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400e9c0 tx=0x7fe0f400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.235+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400ccc0 con 0x7fe108078290 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.235+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f40054d0 con 0x7fe108078290 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.235+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4010430 con 0x7fe108078290 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.235+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1081a6d50 con 0x7fe108078290 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.235+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1081ad4b0 con 0x7fe108078290 2024-09-21T01:22:23.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.236+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe108077d50 con 0x7fe108078290 2024-09-21T01:22:23.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.239+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7fe0f4004f20 con 0x7fe108078290 2024-09-21T01:22:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.240+0000 7fe0fea00640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0ec0b6700 0x7fe0ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.241+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe0f40e8070 con 0x7fe108078290 2024-09-21T01:22:23.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.241+0000 7fe105e00640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0ec0b6700 0x7fe0ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:23.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.241+0000 7fe105e00640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0ec0b6700 0x7fe0ec0b8c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005ab0 tx=0x7fe0f8005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:23.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.242+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0f4056710 con 0x7fe108078290 2024-09-21T01:22:23.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.439+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe108063800 con 0x7fe0ec0b6700 2024-09-21T01:22:23.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.445+0000 7fe0fea00640 1 -- 172.21.15.27:0/2227341156 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340597 (secure 0 0 0) 0x7fe108063800 con 0x7fe0ec0b6700 2024-09-21T01:22:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:23.451 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:22:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0ec0b6700 msgr2=0x7fe0ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0ec0b6700 0x7fe0ec0b8c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005ab0 tx=0x7fe0f8005a20 comp rx=0 tx=0).stop 2024-09-21T01:22:23.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 msgr2=0x7fe1081a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 0x7fe1081a1090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400e9c0 tx=0x7fe0f400edc0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 shutdown_connections 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0ec0b6700 0x7fe0ec0b8c10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe108114740 0x7fe1081a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe108078290 0x7fe1081a1090 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 --2- 172.21.15.27:0/2227341156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe108077940 0x7fe1081a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 >> 172.21.15.27:0/2227341156 conn(0x7fe1081017d0 msgr2=0x7fe108116840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 shutdown_connections 2024-09-21T01:22:23.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:23.450+0000 7fe10d200640 1 -- 172.21.15.27:0/2227341156 wait complete. 2024-09-21T01:22:23.850 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:23 smithi027 ceph-mon[30375]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 55 op/s 2024-09-21T01:22:24.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:23 smithi089 ceph-mon[34486]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 55 op/s 2024-09-21T01:22:24.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:23 smithi098 ceph-mon[34372]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 55 op/s 2024-09-21T01:22:24.291 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-21T01:22:22.476012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1460908,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1761,"num_read_kb":1143,"num_write":618,"num_write_kb":4411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3019356,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373696,"kb_used_data":16008,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124568128,"statfs":{"total":1151940427776,"available":1151557763072,"internally_reserved":0,"allocated":16392192,"data_stored":9986322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":796359,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":434,"num_read_kb":352,"num_write":234,"num_write_kb":2106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004451"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:17.608226+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:22:17.608226+0000","last_peered":"2024-09-21T01:22:17.608226+0000","last_clean":"2024-09-21T01:22:17.608226+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:22:17.608226+0000","last_undegraded":"2024-09-21T01:22:17.608226+0000","last_fullsized":"2024-09-21T01:22:17.608226+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824857+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:21:40.824857+0000","last_peered":"2024-09-21T01:21:40.824857+0000","last_clean":"2024-09-21T01:21:40.824857+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:21:40.824857+0000","last_undegraded":"2024-09-21T01:21:40.824857+0000","last_fullsized":"2024-09-21T01:21:40.824857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166128+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:20:42.166128+0000","last_peered":"2024-09-21T01:20:42.166128+0000","last_clean":"2024-09-21T01:20:42.166128+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:20:42.166128+0000","last_undegraded":"2024-09-21T01:20:42.166128+0000","last_fullsized":"2024-09-21T01:20:42.166128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213160+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:20:42.213160+0000","last_peered":"2024-09-21T01:20:42.213160+0000","last_clean":"2024-09-21T01:20:42.213160+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:20:42.213160+0000","last_undegraded":"2024-09-21T01:20:42.213160+0000","last_fullsized":"2024-09-21T01:20:42.213160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830390+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:21:40.830390+0000","last_peered":"2024-09-21T01:21:40.830390+0000","last_clean":"2024-09-21T01:21:40.830390+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:21:40.830390+0000","last_undegraded":"2024-09-21T01:21:40.830390+0000","last_fullsized":"2024-09-21T01:21:40.830390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168785+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:20:42.168785+0000","last_peered":"2024-09-21T01:20:42.168785+0000","last_clean":"2024-09-21T01:20:42.168785+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:20:42.168785+0000","last_undegraded":"2024-09-21T01:20:42.168785+0000","last_fullsized":"2024-09-21T01:20:42.168785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:37.206045+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:21:37.206045+0000","last_peered":"2024-09-21T01:21:37.206045+0000","last_clean":"2024-09-21T01:21:37.206045+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:21:37.206045+0000","last_undegraded":"2024-09-21T01:21:37.206045+0000","last_fullsized":"2024-09-21T01:21:37.206045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837613+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:21:40.837613+0000","last_peered":"2024-09-21T01:21:40.837613+0000","last_clean":"2024-09-21T01:21:40.837613+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:21:40.837613+0000","last_undegraded":"2024-09-21T01:21:40.837613+0000","last_fullsized":"2024-09-21T01:21:40.837613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169405+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:20:42.169405+0000","last_peered":"2024-09-21T01:20:42.169405+0000","last_clean":"2024-09-21T01:20:42.169405+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:20:42.169405+0000","last_undegraded":"2024-09-21T01:20:42.169405+0000","last_fullsized":"2024-09-21T01:20:42.169405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172455+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:42.172455+0000","last_peered":"2024-09-21T01:20:42.172455+0000","last_clean":"2024-09-21T01:20:42.172455+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:42.172455+0000","last_undegraded":"2024-09-21T01:20:42.172455+0000","last_fullsized":"2024-09-21T01:20:42.172455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.840190+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:21:40.840190+0000","last_peered":"2024-09-21T01:21:40.840190+0000","last_clean":"2024-09-21T01:21:40.840190+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:21:40.840190+0000","last_undegraded":"2024-09-21T01:21:40.840190+0000","last_fullsized":"2024-09-21T01:21:40.840190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169337+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:20:42.169337+0000","last_peered":"2024-09-21T01:20:42.169337+0000","last_clean":"2024-09-21T01:20:42.169337+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:20:42.169337+0000","last_undegraded":"2024-09-21T01:20:42.169337+0000","last_fullsized":"2024-09-21T01:20:42.169337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837797+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:21:40.837797+0000","last_peered":"2024-09-21T01:21:40.837797+0000","last_clean":"2024-09-21T01:21:40.837797+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:21:40.837797+0000","last_undegraded":"2024-09-21T01:21:40.837797+0000","last_fullsized":"2024-09-21T01:21:40.837797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171800+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:42.171800+0000","last_peered":"2024-09-21T01:20:42.171800+0000","last_clean":"2024-09-21T01:20:42.171800+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:42.171800+0000","last_undegraded":"2024-09-21T01:20:42.171800+0000","last_fullsized":"2024-09-21T01:20:42.171800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170156+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:20:42.170156+0000","last_peered":"2024-09-21T01:20:42.170156+0000","last_clean":"2024-09-21T01:20:42.170156+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:20:42.170156+0000","last_undegraded":"2024-09-21T01:20:42.170156+0000","last_fullsized":"2024-09-21T01:20:42.170156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834742+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:21:40.834742+0000","last_peered":"2024-09-21T01:21:40.834742+0000","last_clean":"2024-09-21T01:21:40.834742+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:21:40.834742+0000","last_undegraded":"2024-09-21T01:21:40.834742+0000","last_fullsized":"2024-09-21T01:21:40.834742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166497+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:42.166497+0000","last_peered":"2024-09-21T01:20:42.166497+0000","last_clean":"2024-09-21T01:20:42.166497+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:42.166497+0000","last_undegraded":"2024-09-21T01:20:42.166497+0000","last_fullsized":"2024-09-21T01:20:42.166497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169996+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:20:42.169996+0000","last_peered":"2024-09-21T01:20:42.169996+0000","last_clean":"2024-09-21T01:20:42.169996+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:20:42.169996+0000","last_undegraded":"2024-09-21T01:20:42.169996+0000","last_fullsized":"2024-09-21T01:20:42.169996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.820540+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:21:40.820540+0000","last_peered":"2024-09-21T01:21:40.820540+0000","last_clean":"2024-09-21T01:21:40.820540+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:21:40.820540+0000","last_undegraded":"2024-09-21T01:21:40.820540+0000","last_fullsized":"2024-09-21T01:21:40.820540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212818+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:42.212818+0000","last_peered":"2024-09-21T01:20:42.212818+0000","last_clean":"2024-09-21T01:20:42.212818+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:42.212818+0000","last_undegraded":"2024-09-21T01:20:42.212818+0000","last_fullsized":"2024-09-21T01:20:42.212818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166998+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:20:42.166998+0000","last_peered":"2024-09-21T01:20:42.166998+0000","last_clean":"2024-09-21T01:20:42.166998+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:20:42.166998+0000","last_undegraded":"2024-09-21T01:20:42.166998+0000","last_fullsized":"2024-09-21T01:20:42.166998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.785150+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:21:40.785150+0000","last_peered":"2024-09-21T01:21:40.785150+0000","last_clean":"2024-09-21T01:21:40.785150+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:21:40.785150+0000","last_undegraded":"2024-09-21T01:21:40.785150+0000","last_fullsized":"2024-09-21T01:21:40.785150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173403+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:42.173403+0000","last_peered":"2024-09-21T01:20:42.173403+0000","last_clean":"2024-09-21T01:20:42.173403+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:42.173403+0000","last_undegraded":"2024-09-21T01:20:42.173403+0000","last_fullsized":"2024-09-21T01:20:42.173403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170133+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:20:42.170133+0000","last_peered":"2024-09-21T01:20:42.170133+0000","last_clean":"2024-09-21T01:20:42.170133+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.170133+0000","last_undegraded":"2024-09-21T01:20:42.170133+0000","last_fullsized":"2024-09-21T01:20:42.170133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834120+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:21:40.834120+0000","last_peered":"2024-09-21T01:21:40.834120+0000","last_clean":"2024-09-21T01:21:40.834120+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:21:40.834120+0000","last_undegraded":"2024-09-21T01:21:40.834120+0000","last_fullsized":"2024-09-21T01:21:40.834120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170465+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:42.170465+0000","last_peered":"2024-09-21T01:20:42.170465+0000","last_clean":"2024-09-21T01:20:42.170465+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:42.170465+0000","last_undegraded":"2024-09-21T01:20:42.170465+0000","last_fullsized":"2024-09-21T01:20:42.170465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166302+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:20:42.166302+0000","last_peered":"2024-09-21T01:20:42.166302+0000","last_clean":"2024-09-21T01:20:42.166302+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:20:42.166302+0000","last_undegraded":"2024-09-21T01:20:42.166302+0000","last_fullsized":"2024-09-21T01:20:42.166302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834414+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:21:40.834414+0000","last_peered":"2024-09-21T01:21:40.834414+0000","last_clean":"2024-09-21T01:21:40.834414+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:21:40.834414+0000","last_undegraded":"2024-09-21T01:21:40.834414+0000","last_fullsized":"2024-09-21T01:21:40.834414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'101","reported_seq":232,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.973077+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:22:18.973077+0000","last_peered":"2024-09-21T01:22:18.973077+0000","last_clean":"2024-09-21T01:22:18.973077+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:22:18.973077+0000","last_undegraded":"2024-09-21T01:22:18.973077+0000","last_fullsized":"2024-09-21T01:22:18.973077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":184,"num_write_kb":1223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212484+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:20:42.212484+0000","last_peered":"2024-09-21T01:20:42.212484+0000","last_clean":"2024-09-21T01:20:42.212484+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.212484+0000","last_undegraded":"2024-09-21T01:20:42.212484+0000","last_fullsized":"2024-09-21T01:20:42.212484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.814422+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:21:40.814422+0000","last_peered":"2024-09-21T01:21:40.814422+0000","last_clean":"2024-09-21T01:21:40.814422+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:21:40.814422+0000","last_undegraded":"2024-09-21T01:21:40.814422+0000","last_fullsized":"2024-09-21T01:21:40.814422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.185170+0000","last_change":"2024-09-21T01:20:42.184446+0000","last_active":"2024-09-21T01:20:42.185170+0000","last_peered":"2024-09-21T01:20:42.185170+0000","last_clean":"2024-09-21T01:20:42.185170+0000","last_became_active":"2024-09-21T01:20:42.172338+0000","last_became_peered":"2024-09-21T01:20:42.172338+0000","last_unstale":"2024-09-21T01:20:42.185170+0000","last_undegraded":"2024-09-21T01:20:42.185170+0000","last_fullsized":"2024-09-21T01:20:42.185170+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:12.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168852+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:20:42.168852+0000","last_peered":"2024-09-21T01:20:42.168852+0000","last_clean":"2024-09-21T01:20:42.168852+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:20:42.168852+0000","last_undegraded":"2024-09-21T01:20:42.168852+0000","last_fullsized":"2024-09-21T01:20:42.168852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832976+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:21:40.832976+0000","last_peered":"2024-09-21T01:21:40.832976+0000","last_clean":"2024-09-21T01:21:40.832976+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:21:40.832976+0000","last_undegraded":"2024-09-21T01:21:40.832976+0000","last_fullsized":"2024-09-21T01:21:40.832976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167193+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:20:42.167193+0000","last_peered":"2024-09-21T01:20:42.167193+0000","last_clean":"2024-09-21T01:20:42.167193+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:20:42.167193+0000","last_undegraded":"2024-09-21T01:20:42.167193+0000","last_fullsized":"2024-09-21T01:20:42.167193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171244+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:20:42.171244+0000","last_peered":"2024-09-21T01:20:42.171244+0000","last_clean":"2024-09-21T01:20:42.171244+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:20:42.171244+0000","last_undegraded":"2024-09-21T01:20:42.171244+0000","last_fullsized":"2024-09-21T01:20:42.171244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829906+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:21:40.829906+0000","last_peered":"2024-09-21T01:21:40.829906+0000","last_clean":"2024-09-21T01:21:40.829906+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:21:40.829906+0000","last_undegraded":"2024-09-21T01:21:40.829906+0000","last_fullsized":"2024-09-21T01:21:40.829906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":188,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.611456+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:22:18.611456+0000","last_peered":"2024-09-21T01:22:18.611456+0000","last_clean":"2024-09-21T01:22:18.611456+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:22:18.611456+0000","last_undegraded":"2024-09-21T01:22:18.611456+0000","last_fullsized":"2024-09-21T01:22:18.611456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368522+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:20:42.368522+0000","last_peered":"2024-09-21T01:20:42.368522+0000","last_clean":"2024-09-21T01:20:42.368522+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:20:42.368522+0000","last_undegraded":"2024-09-21T01:20:42.368522+0000","last_fullsized":"2024-09-21T01:20:42.368522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824000+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:21:40.824000+0000","last_peered":"2024-09-21T01:21:40.824000+0000","last_clean":"2024-09-21T01:21:40.824000+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:21:40.824000+0000","last_undegraded":"2024-09-21T01:21:40.824000+0000","last_fullsized":"2024-09-21T01:21:40.824000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":187,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:17.608243+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:22:17.608243+0000","last_peered":"2024-09-21T01:22:17.608243+0000","last_clean":"2024-09-21T01:22:17.608243+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:22:17.608243+0000","last_undegraded":"2024-09-21T01:22:17.608243+0000","last_fullsized":"2024-09-21T01:22:17.608243+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368663+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:20:42.368663+0000","last_peered":"2024-09-21T01:20:42.368663+0000","last_clean":"2024-09-21T01:20:42.368663+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:20:42.368663+0000","last_undegraded":"2024-09-21T01:20:42.368663+0000","last_fullsized":"2024-09-21T01:20:42.368663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.811679+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:21:40.811679+0000","last_peered":"2024-09-21T01:21:40.811679+0000","last_clean":"2024-09-21T01:21:40.811679+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:21:40.811679+0000","last_undegraded":"2024-09-21T01:21:40.811679+0000","last_fullsized":"2024-09-21T01:21:40.811679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166486+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:42.166486+0000","last_peered":"2024-09-21T01:20:42.166486+0000","last_clean":"2024-09-21T01:20:42.166486+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:42.166486+0000","last_undegraded":"2024-09-21T01:20:42.166486+0000","last_fullsized":"2024-09-21T01:20:42.166486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171878+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:20:42.171878+0000","last_peered":"2024-09-21T01:20:42.171878+0000","last_clean":"2024-09-21T01:20:42.171878+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:20:42.171878+0000","last_undegraded":"2024-09-21T01:20:42.171878+0000","last_fullsized":"2024-09-21T01:20:42.171878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837018+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:21:40.837018+0000","last_peered":"2024-09-21T01:21:40.837018+0000","last_clean":"2024-09-21T01:21:40.837018+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:21:40.837018+0000","last_undegraded":"2024-09-21T01:21:40.837018+0000","last_fullsized":"2024-09-21T01:21:40.837018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169791+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:42.169791+0000","last_peered":"2024-09-21T01:20:42.169791+0000","last_clean":"2024-09-21T01:20:42.169791+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:42.169791+0000","last_undegraded":"2024-09-21T01:20:42.169791+0000","last_fullsized":"2024-09-21T01:20:42.169791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166664+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:20:42.166664+0000","last_peered":"2024-09-21T01:20:42.166664+0000","last_clean":"2024-09-21T01:20:42.166664+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:20:42.166664+0000","last_undegraded":"2024-09-21T01:20:42.166664+0000","last_fullsized":"2024-09-21T01:20:42.166664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166645+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:20:42.166645+0000","last_peered":"2024-09-21T01:20:42.166645+0000","last_clean":"2024-09-21T01:20:42.166645+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:20:42.166645+0000","last_undegraded":"2024-09-21T01:20:42.166645+0000","last_fullsized":"2024-09-21T01:20:42.166645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838028+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:21:40.838028+0000","last_peered":"2024-09-21T01:21:40.838028+0000","last_clean":"2024-09-21T01:21:40.838028+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:21:40.838028+0000","last_undegraded":"2024-09-21T01:21:40.838028+0000","last_fullsized":"2024-09-21T01:21:40.838028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.348297+0000","last_change":"2024-09-21T01:20:42.188731+0000","last_active":"2024-09-21T01:22:18.348297+0000","last_peered":"2024-09-21T01:22:18.348297+0000","last_clean":"2024-09-21T01:22:18.348297+0000","last_became_active":"2024-09-21T01:20:42.172041+0000","last_became_peered":"2024-09-21T01:20:42.172041+0000","last_unstale":"2024-09-21T01:22:18.348297+0000","last_undegraded":"2024-09-21T01:22:18.348297+0000","last_fullsized":"2024-09-21T01:22:18.348297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:42.392074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173149+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:20:42.173149+0000","last_peered":"2024-09-21T01:20:42.173149+0000","last_clean":"2024-09-21T01:20:42.173149+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:20:42.173149+0000","last_undegraded":"2024-09-21T01:20:42.173149+0000","last_fullsized":"2024-09-21T01:20:42.173149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831868+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:21:40.831868+0000","last_peered":"2024-09-21T01:21:40.831868+0000","last_clean":"2024-09-21T01:21:40.831868+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:21:40.831868+0000","last_undegraded":"2024-09-21T01:21:40.831868+0000","last_fullsized":"2024-09-21T01:21:40.831868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'62","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.972597+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:22:18.972597+0000","last_peered":"2024-09-21T01:22:18.972597+0000","last_clean":"2024-09-21T01:22:18.972597+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:22:18.972597+0000","last_undegraded":"2024-09-21T01:22:18.972597+0000","last_fullsized":"2024-09-21T01:22:18.972597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171335+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:20:42.171335+0000","last_peered":"2024-09-21T01:20:42.171335+0000","last_clean":"2024-09-21T01:20:42.171335+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:20:42.171335+0000","last_undegraded":"2024-09-21T01:20:42.171335+0000","last_fullsized":"2024-09-21T01:20:42.171335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830380+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:21:40.830380+0000","last_peered":"2024-09-21T01:21:40.830380+0000","last_clean":"2024-09-21T01:21:40.830380+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:21:40.830380+0000","last_undegraded":"2024-09-21T01:21:40.830380+0000","last_fullsized":"2024-09-21T01:21:40.830380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368686+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:42.368686+0000","last_peered":"2024-09-21T01:20:42.368686+0000","last_clean":"2024-09-21T01:20:42.368686+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:42.368686+0000","last_undegraded":"2024-09-21T01:20:42.368686+0000","last_fullsized":"2024-09-21T01:20:42.368686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173156+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:20:42.173156+0000","last_peered":"2024-09-21T01:20:42.173156+0000","last_clean":"2024-09-21T01:20:42.173156+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:20:42.173156+0000","last_undegraded":"2024-09-21T01:20:42.173156+0000","last_fullsized":"2024-09-21T01:20:42.173156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824620+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:21:40.824620+0000","last_peered":"2024-09-21T01:21:40.824620+0000","last_clean":"2024-09-21T01:21:40.824620+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:21:40.824620+0000","last_undegraded":"2024-09-21T01:21:40.824620+0000","last_fullsized":"2024-09-21T01:21:40.824620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.189333+0000","last_change":"2024-09-21T01:20:42.188808+0000","last_active":"2024-09-21T01:20:42.189333+0000","last_peered":"2024-09-21T01:20:42.189333+0000","last_clean":"2024-09-21T01:20:42.189333+0000","last_became_active":"2024-09-21T01:20:42.175348+0000","last_became_peered":"2024-09-21T01:20:42.175348+0000","last_unstale":"2024-09-21T01:20:42.189333+0000","last_undegraded":"2024-09-21T01:20:42.189333+0000","last_fullsized":"2024-09-21T01:20:42.189333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.746569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173579+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:20:42.173579+0000","last_peered":"2024-09-21T01:20:42.173579+0000","last_clean":"2024-09-21T01:20:42.173579+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:20:42.173579+0000","last_undegraded":"2024-09-21T01:20:42.173579+0000","last_fullsized":"2024-09-21T01:20:42.173579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.839227+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:21:40.839227+0000","last_peered":"2024-09-21T01:21:40.839227+0000","last_clean":"2024-09-21T01:21:40.839227+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:21:40.839227+0000","last_undegraded":"2024-09-21T01:21:40.839227+0000","last_fullsized":"2024-09-21T01:21:40.839227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171236+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:42.171236+0000","last_peered":"2024-09-21T01:20:42.171236+0000","last_clean":"2024-09-21T01:20:42.171236+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:42.171236+0000","last_undegraded":"2024-09-21T01:20:42.171236+0000","last_fullsized":"2024-09-21T01:20:42.171236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368646+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:20:42.368646+0000","last_peered":"2024-09-21T01:20:42.368646+0000","last_clean":"2024-09-21T01:20:42.368646+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:20:42.368646+0000","last_undegraded":"2024-09-21T01:20:42.368646+0000","last_fullsized":"2024-09-21T01:20:42.368646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831667+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:21:40.831667+0000","last_peered":"2024-09-21T01:21:40.831667+0000","last_clean":"2024-09-21T01:21:40.831667+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:21:40.831667+0000","last_undegraded":"2024-09-21T01:21:40.831667+0000","last_fullsized":"2024-09-21T01:21:40.831667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":191,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.610592+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:22:18.610592+0000","last_peered":"2024-09-21T01:22:18.610592+0000","last_clean":"2024-09-21T01:22:18.610592+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:22:18.610592+0000","last_undegraded":"2024-09-21T01:22:18.610592+0000","last_fullsized":"2024-09-21T01:22:18.610592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172600+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:20:42.172600+0000","last_peered":"2024-09-21T01:20:42.172600+0000","last_clean":"2024-09-21T01:20:42.172600+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:20:42.172600+0000","last_undegraded":"2024-09-21T01:20:42.172600+0000","last_fullsized":"2024-09-21T01:20:42.172600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832208+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:21:40.832208+0000","last_peered":"2024-09-21T01:21:40.832208+0000","last_clean":"2024-09-21T01:21:40.832208+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:21:40.832208+0000","last_undegraded":"2024-09-21T01:21:40.832208+0000","last_fullsized":"2024-09-21T01:21:40.832208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173980+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:42.173980+0000","last_peered":"2024-09-21T01:20:42.173980+0000","last_clean":"2024-09-21T01:20:42.173980+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:42.173980+0000","last_undegraded":"2024-09-21T01:20:42.173980+0000","last_fullsized":"2024-09-21T01:20:42.173980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168815+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:20:42.168815+0000","last_peered":"2024-09-21T01:20:42.168815+0000","last_clean":"2024-09-21T01:20:42.168815+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:20:42.168815+0000","last_undegraded":"2024-09-21T01:20:42.168815+0000","last_fullsized":"2024-09-21T01:20:42.168815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829263+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:21:40.829263+0000","last_peered":"2024-09-21T01:21:40.829263+0000","last_clean":"2024-09-21T01:21:40.829263+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:21:40.829263+0000","last_undegraded":"2024-09-21T01:21:40.829263+0000","last_fullsized":"2024-09-21T01:21:40.829263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166698+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:42.166698+0000","last_peered":"2024-09-21T01:20:42.166698+0000","last_clean":"2024-09-21T01:20:42.166698+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:42.166698+0000","last_undegraded":"2024-09-21T01:20:42.166698+0000","last_fullsized":"2024-09-21T01:20:42.166698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173990+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:20:42.173990+0000","last_peered":"2024-09-21T01:20:42.173990+0000","last_clean":"2024-09-21T01:20:42.173990+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:20:42.173990+0000","last_undegraded":"2024-09-21T01:20:42.173990+0000","last_fullsized":"2024-09-21T01:20:42.173990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.810339+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:21:40.810339+0000","last_peered":"2024-09-21T01:21:40.810339+0000","last_clean":"2024-09-21T01:21:40.810339+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:21:40.810339+0000","last_undegraded":"2024-09-21T01:21:40.810339+0000","last_fullsized":"2024-09-21T01:21:40.810339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":392,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:20.333237+0000","last_change":"2024-09-21T01:20:42.182111+0000","last_active":"2024-09-21T01:22:20.333237+0000","last_peered":"2024-09-21T01:22:20.333237+0000","last_clean":"2024-09-21T01:22:20.333237+0000","last_became_active":"2024-09-21T01:20:42.174484+0000","last_became_peered":"2024-09-21T01:20:42.174484+0000","last_unstale":"2024-09-21T01:22:20.333237+0000","last_undegraded":"2024-09-21T01:22:20.333237+0000","last_fullsized":"2024-09-21T01:22:20.333237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:45.448208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":281,"num_read_kb":214,"num_write":44,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":66212,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166872+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:20:42.166872+0000","last_peered":"2024-09-21T01:20:42.166872+0000","last_clean":"2024-09-21T01:20:42.166872+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:20:42.166872+0000","last_undegraded":"2024-09-21T01:20:42.166872+0000","last_fullsized":"2024-09-21T01:20:42.166872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838943+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:21:40.838943+0000","last_peered":"2024-09-21T01:21:40.838943+0000","last_clean":"2024-09-21T01:21:40.838943+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:21:40.838943+0000","last_undegraded":"2024-09-21T01:21:40.838943+0000","last_fullsized":"2024-09-21T01:21:40.838943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174291+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:42.174291+0000","last_peered":"2024-09-21T01:20:42.174291+0000","last_clean":"2024-09-21T01:20:42.174291+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:42.174291+0000","last_undegraded":"2024-09-21T01:20:42.174291+0000","last_fullsized":"2024-09-21T01:20:42.174291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169605+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:20:42.169605+0000","last_peered":"2024-09-21T01:20:42.169605+0000","last_clean":"2024-09-21T01:20:42.169605+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:20:42.169605+0000","last_undegraded":"2024-09-21T01:20:42.169605+0000","last_fullsized":"2024-09-21T01:20:42.169605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.822722+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:21:40.822722+0000","last_peered":"2024-09-21T01:21:40.822722+0000","last_clean":"2024-09-21T01:21:40.822722+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:21:40.822722+0000","last_undegraded":"2024-09-21T01:21:40.822722+0000","last_fullsized":"2024-09-21T01:21:40.822722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167148+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:42.167148+0000","last_peered":"2024-09-21T01:20:42.167148+0000","last_clean":"2024-09-21T01:20:42.167148+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:42.167148+0000","last_undegraded":"2024-09-21T01:20:42.167148+0000","last_fullsized":"2024-09-21T01:20:42.167148+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166594+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:20:42.166594+0000","last_peered":"2024-09-21T01:20:42.166594+0000","last_clean":"2024-09-21T01:20:42.166594+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:20:42.166594+0000","last_undegraded":"2024-09-21T01:20:42.166594+0000","last_fullsized":"2024-09-21T01:20:42.166594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837693+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:21:40.837693+0000","last_peered":"2024-09-21T01:21:40.837693+0000","last_clean":"2024-09-21T01:21:40.837693+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:21:40.837693+0000","last_undegraded":"2024-09-21T01:21:40.837693+0000","last_fullsized":"2024-09-21T01:21:40.837693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":322,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:19.612424+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:22:19.612424+0000","last_peered":"2024-09-21T01:22:19.612424+0000","last_clean":"2024-09-21T01:22:19.612424+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:22:19.612424+0000","last_undegraded":"2024-09-21T01:22:19.612424+0000","last_fullsized":"2024-09-21T01:22:19.612424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212589+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:20:42.212589+0000","last_peered":"2024-09-21T01:20:42.212589+0000","last_clean":"2024-09-21T01:20:42.212589+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:20:42.212589+0000","last_undegraded":"2024-09-21T01:20:42.212589+0000","last_fullsized":"2024-09-21T01:20:42.212589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171758+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:20:42.171758+0000","last_peered":"2024-09-21T01:20:42.171758+0000","last_clean":"2024-09-21T01:20:42.171758+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:20:42.171758+0000","last_undegraded":"2024-09-21T01:20:42.171758+0000","last_fullsized":"2024-09-21T01:20:42.171758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.827678+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:21:40.827678+0000","last_peered":"2024-09-21T01:21:40.827678+0000","last_clean":"2024-09-21T01:21:40.827678+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:21:40.827678+0000","last_undegraded":"2024-09-21T01:21:40.827678+0000","last_fullsized":"2024-09-21T01:21:40.827678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173442+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:42.173442+0000","last_peered":"2024-09-21T01:20:42.173442+0000","last_clean":"2024-09-21T01:20:42.173442+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:42.173442+0000","last_undegraded":"2024-09-21T01:20:42.173442+0000","last_fullsized":"2024-09-21T01:20:42.173442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169158+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:20:42.169158+0000","last_peered":"2024-09-21T01:20:42.169158+0000","last_clean":"2024-09-21T01:20:42.169158+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:20:42.169158+0000","last_undegraded":"2024-09-21T01:20:42.169158+0000","last_fullsized":"2024-09-21T01:20:42.169158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833231+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:21:40.833231+0000","last_peered":"2024-09-21T01:21:40.833231+0000","last_clean":"2024-09-21T01:21:40.833231+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:21:40.833231+0000","last_undegraded":"2024-09-21T01:21:40.833231+0000","last_fullsized":"2024-09-21T01:21:40.833231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368185+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:42.368185+0000","last_peered":"2024-09-21T01:20:42.368185+0000","last_clean":"2024-09-21T01:20:42.368185+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:42.368185+0000","last_undegraded":"2024-09-21T01:20:42.368185+0000","last_fullsized":"2024-09-21T01:20:42.368185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169957+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:42.169957+0000","last_peered":"2024-09-21T01:20:42.169957+0000","last_clean":"2024-09-21T01:20:42.169957+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:42.169957+0000","last_undegraded":"2024-09-21T01:20:42.169957+0000","last_fullsized":"2024-09-21T01:20:42.169957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833060+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:21:40.833060+0000","last_peered":"2024-09-21T01:21:40.833060+0000","last_clean":"2024-09-21T01:21:40.833060+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:21:40.833060+0000","last_undegraded":"2024-09-21T01:21:40.833060+0000","last_fullsized":"2024-09-21T01:21:40.833060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166300+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:20:42.166300+0000","last_peered":"2024-09-21T01:20:42.166300+0000","last_clean":"2024-09-21T01:20:42.166300+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:20:42.166300+0000","last_undegraded":"2024-09-21T01:20:42.166300+0000","last_fullsized":"2024-09-21T01:20:42.166300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":195,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:20.614791+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:22:20.614791+0000","last_peered":"2024-09-21T01:22:20.614791+0000","last_clean":"2024-09-21T01:22:20.614791+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:22:20.614791+0000","last_undegraded":"2024-09-21T01:22:20.614791+0000","last_fullsized":"2024-09-21T01:22:20.614791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1465,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.812341+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:21:40.812341+0000","last_peered":"2024-09-21T01:21:40.812341+0000","last_clean":"2024-09-21T01:21:40.812341+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:21:40.812341+0000","last_undegraded":"2024-09-21T01:21:40.812341+0000","last_fullsized":"2024-09-21T01:21:40.812341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166566+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:20:42.166566+0000","last_peered":"2024-09-21T01:20:42.166566+0000","last_clean":"2024-09-21T01:20:42.166566+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:20:42.166566+0000","last_undegraded":"2024-09-21T01:20:42.166566+0000","last_fullsized":"2024-09-21T01:20:42.166566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.825083+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:21:40.825083+0000","last_peered":"2024-09-21T01:21:40.825083+0000","last_clean":"2024-09-21T01:21:40.825083+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:21:40.825083+0000","last_undegraded":"2024-09-21T01:21:40.825083+0000","last_fullsized":"2024-09-21T01:21:40.825083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.823972+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:21:40.823972+0000","last_peered":"2024-09-21T01:21:40.823972+0000","last_clean":"2024-09-21T01:21:40.823972+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:21:40.823972+0000","last_undegraded":"2024-09-21T01:21:40.823972+0000","last_fullsized":"2024-09-21T01:21:40.823972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831722+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:21:40.831722+0000","last_peered":"2024-09-21T01:21:40.831722+0000","last_clean":"2024-09-21T01:21:40.831722+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:21:40.831722+0000","last_undegraded":"2024-09-21T01:21:40.831722+0000","last_fullsized":"2024-09-21T01:21:40.831722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830027+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:21:40.830027+0000","last_peered":"2024-09-21T01:21:40.830027+0000","last_clean":"2024-09-21T01:21:40.830027+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:21:40.830027+0000","last_undegraded":"2024-09-21T01:21:40.830027+0000","last_fullsized":"2024-09-21T01:21:40.830027+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829116+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:21:40.829116+0000","last_peered":"2024-09-21T01:21:40.829116+0000","last_clean":"2024-09-21T01:21:40.829116+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:21:40.829116+0000","last_undegraded":"2024-09-21T01:21:40.829116+0000","last_fullsized":"2024-09-21T01:21:40.829116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824234+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:21:40.824234+0000","last_peered":"2024-09-21T01:21:40.824234+0000","last_clean":"2024-09-21T01:21:40.824234+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:21:40.824234+0000","last_undegraded":"2024-09-21T01:21:40.824234+0000","last_fullsized":"2024-09-21T01:21:40.824234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831569+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:21:40.831569+0000","last_peered":"2024-09-21T01:21:40.831569+0000","last_clean":"2024-09-21T01:21:40.831569+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:21:40.831569+0000","last_undegraded":"2024-09-21T01:21:40.831569+0000","last_fullsized":"2024-09-21T01:21:40.831569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832623+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:21:40.832623+0000","last_peered":"2024-09-21T01:21:40.832623+0000","last_clean":"2024-09-21T01:21:40.832623+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:21:40.832623+0000","last_undegraded":"2024-09-21T01:21:40.832623+0000","last_fullsized":"2024-09-21T01:21:40.832623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832822+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:21:40.832822+0000","last_peered":"2024-09-21T01:21:40.832822+0000","last_clean":"2024-09-21T01:21:40.832822+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:21:40.832822+0000","last_undegraded":"2024-09-21T01:21:40.832822+0000","last_fullsized":"2024-09-21T01:21:40.832822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833468+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:21:40.833468+0000","last_peered":"2024-09-21T01:21:40.833468+0000","last_clean":"2024-09-21T01:21:40.833468+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:21:40.833468+0000","last_undegraded":"2024-09-21T01:21:40.833468+0000","last_fullsized":"2024-09-21T01:21:40.833468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838351+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:21:40.838351+0000","last_peered":"2024-09-21T01:21:40.838351+0000","last_clean":"2024-09-21T01:21:40.838351+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:21:40.838351+0000","last_undegraded":"2024-09-21T01:21:40.838351+0000","last_fullsized":"2024-09-21T01:21:40.838351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830634+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:21:40.830634+0000","last_peered":"2024-09-21T01:21:40.830634+0000","last_clean":"2024-09-21T01:21:40.830634+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:21:40.830634+0000","last_undegraded":"2024-09-21T01:21:40.830634+0000","last_fullsized":"2024-09-21T01:21:40.830634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.823049+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:21:40.823049+0000","last_peered":"2024-09-21T01:21:40.823049+0000","last_clean":"2024-09-21T01:21:40.823049+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:21:40.823049+0000","last_undegraded":"2024-09-21T01:21:40.823049+0000","last_fullsized":"2024-09-21T01:21:40.823049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834378+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:21:40.834378+0000","last_peered":"2024-09-21T01:21:40.834378+0000","last_clean":"2024-09-21T01:21:40.834378+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:21:40.834378+0000","last_undegraded":"2024-09-21T01:21:40.834378+0000","last_fullsized":"2024-09-21T01:21:40.834378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837211+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:21:40.837211+0000","last_peered":"2024-09-21T01:21:40.837211+0000","last_clean":"2024-09-21T01:21:40.837211+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:21:40.837211+0000","last_undegraded":"2024-09-21T01:21:40.837211+0000","last_fullsized":"2024-09-21T01:21:40.837211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830574+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:21:40.830574+0000","last_peered":"2024-09-21T01:21:40.830574+0000","last_clean":"2024-09-21T01:21:40.830574+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:21:40.830574+0000","last_undegraded":"2024-09-21T01:21:40.830574+0000","last_fullsized":"2024-09-21T01:21:40.830574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.835525+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:21:40.835525+0000","last_peered":"2024-09-21T01:21:40.835525+0000","last_clean":"2024-09-21T01:21:40.835525+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:21:40.835525+0000","last_undegraded":"2024-09-21T01:21:40.835525+0000","last_fullsized":"2024-09-21T01:21:40.835525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.828196+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:21:40.828196+0000","last_peered":"2024-09-21T01:21:40.828196+0000","last_clean":"2024-09-21T01:21:40.828196+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:21:40.828196+0000","last_undegraded":"2024-09-21T01:21:40.828196+0000","last_fullsized":"2024-09-21T01:21:40.828196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.827534+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:21:40.827534+0000","last_peered":"2024-09-21T01:21:40.827534+0000","last_clean":"2024-09-21T01:21:40.827534+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:21:40.827534+0000","last_undegraded":"2024-09-21T01:21:40.827534+0000","last_fullsized":"2024-09-21T01:21:40.827534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829445+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:21:40.829445+0000","last_peered":"2024-09-21T01:21:40.829445+0000","last_clean":"2024-09-21T01:21:40.829445+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:21:40.829445+0000","last_undegraded":"2024-09-21T01:21:40.829445+0000","last_fullsized":"2024-09-21T01:21:40.829445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.810535+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:21:40.810535+0000","last_peered":"2024-09-21T01:21:40.810535+0000","last_clean":"2024-09-21T01:21:40.810535+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:21:40.810535+0000","last_undegraded":"2024-09-21T01:21:40.810535+0000","last_fullsized":"2024-09-21T01:21:40.810535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838099+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:21:40.838099+0000","last_peered":"2024-09-21T01:21:40.838099+0000","last_clean":"2024-09-21T01:21:40.838099+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:21:40.838099+0000","last_undegraded":"2024-09-21T01:21:40.838099+0000","last_fullsized":"2024-09-21T01:21:40.838099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.825971+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:21:40.825971+0000","last_peered":"2024-09-21T01:21:40.825971+0000","last_clean":"2024-09-21T01:21:40.825971+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:21:40.825971+0000","last_undegraded":"2024-09-21T01:21:40.825971+0000","last_fullsized":"2024-09-21T01:21:40.825971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837719+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:21:40.837719+0000","last_peered":"2024-09-21T01:21:40.837719+0000","last_clean":"2024-09-21T01:21:40.837719+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:21:40.837719+0000","last_undegraded":"2024-09-21T01:21:40.837719+0000","last_fullsized":"2024-09-21T01:21:40.837719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834644+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:21:40.834644+0000","last_peered":"2024-09-21T01:21:40.834644+0000","last_clean":"2024-09-21T01:21:40.834644+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:21:40.834644+0000","last_undegraded":"2024-09-21T01:21:40.834644+0000","last_fullsized":"2024-09-21T01:21:40.834644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830668+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:21:40.830668+0000","last_peered":"2024-09-21T01:21:40.830668+0000","last_clean":"2024-09-21T01:21:40.830668+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:21:40.830668+0000","last_undegraded":"2024-09-21T01:21:40.830668+0000","last_fullsized":"2024-09-21T01:21:40.830668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.826190+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:21:40.826190+0000","last_peered":"2024-09-21T01:21:40.826190+0000","last_clean":"2024-09-21T01:21:40.826190+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:21:40.826190+0000","last_undegraded":"2024-09-21T01:21:40.826190+0000","last_fullsized":"2024-09-21T01:21:40.826190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.821398+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:21:40.821398+0000","last_peered":"2024-09-21T01:21:40.821398+0000","last_clean":"2024-09-21T01:21:40.821398+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:21:40.821398+0000","last_undegraded":"2024-09-21T01:21:40.821398+0000","last_fullsized":"2024-09-21T01:21:40.821398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.817572+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:21:40.817572+0000","last_peered":"2024-09-21T01:21:40.817572+0000","last_clean":"2024-09-21T01:21:40.817572+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:21:40.817572+0000","last_undegraded":"2024-09-21T01:21:40.817572+0000","last_fullsized":"2024-09-21T01:21:40.817572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830525+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:21:40.830525+0000","last_peered":"2024-09-21T01:21:40.830525+0000","last_clean":"2024-09-21T01:21:40.830525+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:21:40.830525+0000","last_undegraded":"2024-09-21T01:21:40.830525+0000","last_fullsized":"2024-09-21T01:21:40.830525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.803325+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:21:40.803325+0000","last_peered":"2024-09-21T01:21:40.803325+0000","last_clean":"2024-09-21T01:21:40.803325+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:21:40.803325+0000","last_undegraded":"2024-09-21T01:21:40.803325+0000","last_fullsized":"2024-09-21T01:21:40.803325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830302+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:21:40.830302+0000","last_peered":"2024-09-21T01:21:40.830302+0000","last_clean":"2024-09-21T01:21:40.830302+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:21:40.830302+0000","last_undegraded":"2024-09-21T01:21:40.830302+0000","last_fullsized":"2024-09-21T01:21:40.830302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":870306,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1680,"num_read_kb":1074,"num_write":493,"num_write_kb":2259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":67516,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2195456,"data_stored":1823926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":375,"ondisk_log_size":375,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":535509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874299,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29192,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715960,"statfs":{"total":95995035648,"available":95965143040,"internally_reserved":0,"allocated":2158592,"data_stored":1595721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037824,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1466368,"data_stored":928521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234054,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1449984,"data_stored":928765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430284,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1133555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593810,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790041,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29108,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716044,"statfs":{"total":95995035648,"available":95965229056,"internally_reserved":0,"allocated":2072576,"data_stored":1521990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":609191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149794,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":530517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378729,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":531732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542254,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":607356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705780,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":532265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":77269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":77528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":471170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:22:24.296 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T01:22:24.296 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T01:22:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:24.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:24 smithi027 ceph-mon[30375]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:25.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:24 smithi089 ceph-mon[34486]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:25.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:24 smithi098 ceph-mon[34372]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 -- 172.21.15.27:0/2709375255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81039b0 msgr2=0x7f27b8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 --2- 172.21.15.27:0/2709375255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81039b0 0x7f27b8103dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009900 tx=0x7f27ac02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 -- 172.21.15.27:0/2709375255 shutdown_connections 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 --2- 172.21.15.27:0/2709375255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 0x7f27b8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 --2- 172.21.15.27:0/2709375255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81092f0 0x7f27b8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 --2- 172.21.15.27:0/2709375255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81039b0 0x7f27b8103dc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.487+0000 7f27be800640 1 -- 172.21.15.27:0/2709375255 >> 172.21.15.27:0/2709375255 conn(0x7f27b80ff400 msgr2=0x7f27b8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.488+0000 7f27be800640 1 -- 172.21.15.27:0/2709375255 shutdown_connections 2024-09-21T01:22:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.488+0000 7f27be800640 1 -- 172.21.15.27:0/2709375255 wait complete. 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 Processor -- start 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 -- start start 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81039b0 0x7f27b819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81092f0 0x7f27b819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 0x7f27b81a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8113950 con 0x7f27b81092f0 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81137d0 con 0x7f27b81039b0 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.489+0000 7f27be800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8113ad0 con 0x7f27b8109c40 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81092f0 0x7f27b819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81039b0 0x7f27b819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bd400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81039b0 0x7f27b819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:59802/0 (socket says 172.21.15.27:59802) 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bd400640 1 -- 172.21.15.27:0/455885139 learned_addr learned my addr 172.21.15.27:0/455885139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bde00640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 0x7f27b81a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bca00640 1 -- 172.21.15.27:0/455885139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 msgr2=0x7f27b81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bca00640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 0x7f27b81a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bca00640 1 -- 172.21.15.27:0/455885139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81039b0 msgr2=0x7f27b819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bca00640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81039b0 0x7f27b819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bca00640 1 -- 172.21.15.27:0/455885139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b81a2490 con 0x7f27b81092f0 2024-09-21T01:22:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.490+0000 7f27bde00640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 0x7f27b81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.491+0000 7f27bca00640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81092f0 0x7f27b819cac0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f27b806b930 tx=0x7f27a400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.491+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a400cd00 con 0x7f27b81092f0 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.491+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b81a2780 con 0x7f27b81092f0 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.491+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a40035f0 con 0x7f27b81092f0 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.491+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4010690 con 0x7f27b81092f0 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.491+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a8ee0 con 0x7f27b81092f0 2024-09-21T01:22:25.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.493+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b8103dc0 con 0x7f27b81092f0 2024-09-21T01:22:25.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.495+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f27a40037a0 con 0x7f27b81092f0 2024-09-21T01:22:25.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.497+0000 7f27b6000640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f27a00b6680 0x7f27a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:25.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.497+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f27a40e7d20 con 0x7f27b81092f0 2024-09-21T01:22:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.497+0000 7f27bd400640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f27a00b6680 0x7f27a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.498+0000 7f27bd400640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f27a00b6680 0x7f27a00b8b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0096c0 tx=0x7f27ac005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:25.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.501+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27a40adea0 con 0x7f27b81092f0 2024-09-21T01:22:25.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.700+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27b810f720 con 0x7f27a00b6680 2024-09-21T01:22:25.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.706+0000 7f27b6000640 1 -- 172.21.15.27:0/455885139 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340603 (secure 0 0 0) 0x7f27b810f720 con 0x7f27a00b6680 2024-09-21T01:22:25.711 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:25.714 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f27a00b6680 msgr2=0x7f27a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f27a00b6680 0x7f27a00b8b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0096c0 tx=0x7f27ac005a20 comp rx=0 tx=0).stop 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81092f0 msgr2=0x7f27b819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81092f0 0x7f27b819cac0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f27b806b930 tx=0x7f27a400ee50 comp rx=0 tx=0).stop 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 shutdown_connections 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f27a00b6680 0x7f27a00b8b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27b8109c40 0x7f27b81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b81092f0 0x7f27b819cac0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 --2- 172.21.15.27:0/455885139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f27b81039b0 0x7f27b819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.711+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 >> 172.21.15.27:0/455885139 conn(0x7f27b80ff400 msgr2=0x7f27b8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.712+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 shutdown_connections 2024-09-21T01:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:25.712+0000 7f27be800640 1 -- 172.21.15.27:0/455885139 wait complete. 2024-09-21T01:22:25.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:25 smithi089 ceph-mon[34486]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T01:22:26.119 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:25 smithi027 ceph-mon[30375]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T01:22:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:25 smithi098 ceph-mon[34372]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T01:22:26.558 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-21T01:22:24.476532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1460908,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1781,"num_read_kb":1162,"num_write":622,"num_write_kb":4411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3019356,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":971,"ondisk_log_size":971,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378688,"kb_used_data":16780,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124563136,"statfs":{"total":1151940427776,"available":1151552651264,"internally_reserved":0,"allocated":17182720,"data_stored":10773314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":796359,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":449,"num_read_kb":371,"num_write":238,"num_write_kb":2106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004260"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":199,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:22.616900+0000","last_change":"2024-09-21T01:19:03.689302+0000","last_active":"2024-09-21T01:22:22.616900+0000","last_peered":"2024-09-21T01:22:22.616900+0000","last_clean":"2024-09-21T01:22:22.616900+0000","last_became_active":"2024-09-21T01:19:03.687868+0000","last_became_peered":"2024-09-21T01:19:03.687868+0000","last_unstale":"2024-09-21T01:22:22.616900+0000","last_undegraded":"2024-09-21T01:22:22.616900+0000","last_fullsized":"2024-09-21T01:22:22.616900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:58:42.778660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824857+0000","last_change":"2024-09-21T01:19:41.901457+0000","last_active":"2024-09-21T01:21:40.824857+0000","last_peered":"2024-09-21T01:21:40.824857+0000","last_clean":"2024-09-21T01:21:40.824857+0000","last_became_active":"2024-09-21T01:19:41.901080+0000","last_became_peered":"2024-09-21T01:19:41.901080+0000","last_unstale":"2024-09-21T01:21:40.824857+0000","last_undegraded":"2024-09-21T01:21:40.824857+0000","last_fullsized":"2024-09-21T01:21:40.824857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:52:49.395226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166128+0000","last_change":"2024-09-21T01:19:16.814242+0000","last_active":"2024-09-21T01:20:42.166128+0000","last_peered":"2024-09-21T01:20:42.166128+0000","last_clean":"2024-09-21T01:20:42.166128+0000","last_became_active":"2024-09-21T01:19:16.813719+0000","last_became_peered":"2024-09-21T01:19:16.813719+0000","last_unstale":"2024-09-21T01:20:42.166128+0000","last_undegraded":"2024-09-21T01:20:42.166128+0000","last_fullsized":"2024-09-21T01:20:42.166128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:44.041329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.213160+0000","last_change":"2024-09-21T01:19:03.694683+0000","last_active":"2024-09-21T01:20:42.213160+0000","last_peered":"2024-09-21T01:20:42.213160+0000","last_clean":"2024-09-21T01:20:42.213160+0000","last_became_active":"2024-09-21T01:19:03.693607+0000","last_became_peered":"2024-09-21T01:19:03.693607+0000","last_unstale":"2024-09-21T01:20:42.213160+0000","last_undegraded":"2024-09-21T01:20:42.213160+0000","last_fullsized":"2024-09-21T01:20:42.213160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:55:10.324109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830390+0000","last_change":"2024-09-21T01:19:06.723624+0000","last_active":"2024-09-21T01:21:40.830390+0000","last_peered":"2024-09-21T01:21:40.830390+0000","last_clean":"2024-09-21T01:21:40.830390+0000","last_became_active":"2024-09-21T01:19:06.722871+0000","last_became_peered":"2024-09-21T01:19:06.722871+0000","last_unstale":"2024-09-21T01:21:40.830390+0000","last_undegraded":"2024-09-21T01:21:40.830390+0000","last_fullsized":"2024-09-21T01:21:40.830390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:05.272754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168785+0000","last_change":"2024-09-21T01:19:16.807893+0000","last_active":"2024-09-21T01:20:42.168785+0000","last_peered":"2024-09-21T01:20:42.168785+0000","last_clean":"2024-09-21T01:20:42.168785+0000","last_became_active":"2024-09-21T01:19:16.807172+0000","last_became_peered":"2024-09-21T01:19:16.807172+0000","last_unstale":"2024-09-21T01:20:42.168785+0000","last_undegraded":"2024-09-21T01:20:42.168785+0000","last_fullsized":"2024-09-21T01:20:42.168785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:58.886344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'13","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:37.206045+0000","last_change":"2024-09-21T01:19:03.685121+0000","last_active":"2024-09-21T01:21:37.206045+0000","last_peered":"2024-09-21T01:21:37.206045+0000","last_clean":"2024-09-21T01:21:37.206045+0000","last_became_active":"2024-09-21T01:19:03.684600+0000","last_became_peered":"2024-09-21T01:19:03.684600+0000","last_unstale":"2024-09-21T01:21:37.206045+0000","last_undegraded":"2024-09-21T01:21:37.206045+0000","last_fullsized":"2024-09-21T01:21:37.206045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:51.644284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837613+0000","last_change":"2024-09-21T01:19:06.711592+0000","last_active":"2024-09-21T01:21:40.837613+0000","last_peered":"2024-09-21T01:21:40.837613+0000","last_clean":"2024-09-21T01:21:40.837613+0000","last_became_active":"2024-09-21T01:19:06.711201+0000","last_became_peered":"2024-09-21T01:19:06.711201+0000","last_unstale":"2024-09-21T01:21:40.837613+0000","last_undegraded":"2024-09-21T01:21:40.837613+0000","last_fullsized":"2024-09-21T01:21:40.837613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:29.376504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169405+0000","last_change":"2024-09-21T01:19:16.820826+0000","last_active":"2024-09-21T01:20:42.169405+0000","last_peered":"2024-09-21T01:20:42.169405+0000","last_clean":"2024-09-21T01:20:42.169405+0000","last_became_active":"2024-09-21T01:19:16.820323+0000","last_became_peered":"2024-09-21T01:19:16.820323+0000","last_unstale":"2024-09-21T01:20:42.169405+0000","last_undegraded":"2024-09-21T01:20:42.169405+0000","last_fullsized":"2024-09-21T01:20:42.169405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:56.831164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172455+0000","last_change":"2024-09-21T01:19:03.681408+0000","last_active":"2024-09-21T01:20:42.172455+0000","last_peered":"2024-09-21T01:20:42.172455+0000","last_clean":"2024-09-21T01:20:42.172455+0000","last_became_active":"2024-09-21T01:19:03.681020+0000","last_became_peered":"2024-09-21T01:19:03.681020+0000","last_unstale":"2024-09-21T01:20:42.172455+0000","last_undegraded":"2024-09-21T01:20:42.172455+0000","last_fullsized":"2024-09-21T01:20:42.172455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:38.027910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.840190+0000","last_change":"2024-09-21T01:19:42.179088+0000","last_active":"2024-09-21T01:21:40.840190+0000","last_peered":"2024-09-21T01:21:40.840190+0000","last_clean":"2024-09-21T01:21:40.840190+0000","last_became_active":"2024-09-21T01:19:42.178720+0000","last_became_peered":"2024-09-21T01:19:42.178720+0000","last_unstale":"2024-09-21T01:21:40.840190+0000","last_undegraded":"2024-09-21T01:21:40.840190+0000","last_fullsized":"2024-09-21T01:21:40.840190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:27:22.611711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169337+0000","last_change":"2024-09-21T01:19:16.808157+0000","last_active":"2024-09-21T01:20:42.169337+0000","last_peered":"2024-09-21T01:20:42.169337+0000","last_clean":"2024-09-21T01:20:42.169337+0000","last_became_active":"2024-09-21T01:19:16.807500+0000","last_became_peered":"2024-09-21T01:19:16.807500+0000","last_unstale":"2024-09-21T01:20:42.169337+0000","last_undegraded":"2024-09-21T01:20:42.169337+0000","last_fullsized":"2024-09-21T01:20:42.169337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.828639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837797+0000","last_change":"2024-09-21T01:19:06.716151+0000","last_active":"2024-09-21T01:21:40.837797+0000","last_peered":"2024-09-21T01:21:40.837797+0000","last_clean":"2024-09-21T01:21:40.837797+0000","last_became_active":"2024-09-21T01:19:06.715704+0000","last_became_peered":"2024-09-21T01:19:06.715704+0000","last_unstale":"2024-09-21T01:21:40.837797+0000","last_undegraded":"2024-09-21T01:21:40.837797+0000","last_fullsized":"2024-09-21T01:21:40.837797+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.411363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171800+0000","last_change":"2024-09-21T01:19:03.685635+0000","last_active":"2024-09-21T01:20:42.171800+0000","last_peered":"2024-09-21T01:20:42.171800+0000","last_clean":"2024-09-21T01:20:42.171800+0000","last_became_active":"2024-09-21T01:19:03.685069+0000","last_became_peered":"2024-09-21T01:19:03.685069+0000","last_unstale":"2024-09-21T01:20:42.171800+0000","last_undegraded":"2024-09-21T01:20:42.171800+0000","last_fullsized":"2024-09-21T01:20:42.171800+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:15.204778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170156+0000","last_change":"2024-09-21T01:19:17.248401+0000","last_active":"2024-09-21T01:20:42.170156+0000","last_peered":"2024-09-21T01:20:42.170156+0000","last_clean":"2024-09-21T01:20:42.170156+0000","last_became_active":"2024-09-21T01:19:17.247720+0000","last_became_peered":"2024-09-21T01:19:17.247720+0000","last_unstale":"2024-09-21T01:20:42.170156+0000","last_undegraded":"2024-09-21T01:20:42.170156+0000","last_fullsized":"2024-09-21T01:20:42.170156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:23.403900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834742+0000","last_change":"2024-09-21T01:19:06.717581+0000","last_active":"2024-09-21T01:21:40.834742+0000","last_peered":"2024-09-21T01:21:40.834742+0000","last_clean":"2024-09-21T01:21:40.834742+0000","last_became_active":"2024-09-21T01:19:06.717262+0000","last_became_peered":"2024-09-21T01:19:06.717262+0000","last_unstale":"2024-09-21T01:21:40.834742+0000","last_undegraded":"2024-09-21T01:21:40.834742+0000","last_fullsized":"2024-09-21T01:21:40.834742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.106444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166497+0000","last_change":"2024-09-21T01:19:03.687034+0000","last_active":"2024-09-21T01:20:42.166497+0000","last_peered":"2024-09-21T01:20:42.166497+0000","last_clean":"2024-09-21T01:20:42.166497+0000","last_became_active":"2024-09-21T01:19:03.686513+0000","last_became_peered":"2024-09-21T01:19:03.686513+0000","last_unstale":"2024-09-21T01:20:42.166497+0000","last_undegraded":"2024-09-21T01:20:42.166497+0000","last_fullsized":"2024-09-21T01:20:42.166497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:46:53.040901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169996+0000","last_change":"2024-09-21T01:19:41.897659+0000","last_active":"2024-09-21T01:20:42.169996+0000","last_peered":"2024-09-21T01:20:42.169996+0000","last_clean":"2024-09-21T01:20:42.169996+0000","last_became_active":"2024-09-21T01:19:41.897067+0000","last_became_peered":"2024-09-21T01:19:41.897067+0000","last_unstale":"2024-09-21T01:20:42.169996+0000","last_undegraded":"2024-09-21T01:20:42.169996+0000","last_fullsized":"2024-09-21T01:20:42.169996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:09:33.892593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.820540+0000","last_change":"2024-09-21T01:19:06.712900+0000","last_active":"2024-09-21T01:21:40.820540+0000","last_peered":"2024-09-21T01:21:40.820540+0000","last_clean":"2024-09-21T01:21:40.820540+0000","last_became_active":"2024-09-21T01:19:06.712487+0000","last_became_peered":"2024-09-21T01:19:06.712487+0000","last_unstale":"2024-09-21T01:21:40.820540+0000","last_undegraded":"2024-09-21T01:21:40.820540+0000","last_fullsized":"2024-09-21T01:21:40.820540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:27.727207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212818+0000","last_change":"2024-09-21T01:19:03.684715+0000","last_active":"2024-09-21T01:20:42.212818+0000","last_peered":"2024-09-21T01:20:42.212818+0000","last_clean":"2024-09-21T01:20:42.212818+0000","last_became_active":"2024-09-21T01:19:03.681319+0000","last_became_peered":"2024-09-21T01:19:03.681319+0000","last_unstale":"2024-09-21T01:20:42.212818+0000","last_undegraded":"2024-09-21T01:20:42.212818+0000","last_fullsized":"2024-09-21T01:20:42.212818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:22.010284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166998+0000","last_change":"2024-09-21T01:19:16.824559+0000","last_active":"2024-09-21T01:20:42.166998+0000","last_peered":"2024-09-21T01:20:42.166998+0000","last_clean":"2024-09-21T01:20:42.166998+0000","last_became_active":"2024-09-21T01:19:16.821405+0000","last_became_peered":"2024-09-21T01:19:16.821405+0000","last_unstale":"2024-09-21T01:20:42.166998+0000","last_undegraded":"2024-09-21T01:20:42.166998+0000","last_fullsized":"2024-09-21T01:20:42.166998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:29:42.803969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.785150+0000","last_change":"2024-09-21T01:19:06.713086+0000","last_active":"2024-09-21T01:21:40.785150+0000","last_peered":"2024-09-21T01:21:40.785150+0000","last_clean":"2024-09-21T01:21:40.785150+0000","last_became_active":"2024-09-21T01:19:06.712690+0000","last_became_peered":"2024-09-21T01:19:06.712690+0000","last_unstale":"2024-09-21T01:21:40.785150+0000","last_undegraded":"2024-09-21T01:21:40.785150+0000","last_fullsized":"2024-09-21T01:21:40.785150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:16.810494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173403+0000","last_change":"2024-09-21T01:19:03.687073+0000","last_active":"2024-09-21T01:20:42.173403+0000","last_peered":"2024-09-21T01:20:42.173403+0000","last_clean":"2024-09-21T01:20:42.173403+0000","last_became_active":"2024-09-21T01:19:03.686672+0000","last_became_peered":"2024-09-21T01:19:03.686672+0000","last_unstale":"2024-09-21T01:20:42.173403+0000","last_undegraded":"2024-09-21T01:20:42.173403+0000","last_fullsized":"2024-09-21T01:20:42.173403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:31.652579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170133+0000","last_change":"2024-09-21T01:19:17.248846+0000","last_active":"2024-09-21T01:20:42.170133+0000","last_peered":"2024-09-21T01:20:42.170133+0000","last_clean":"2024-09-21T01:20:42.170133+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.170133+0000","last_undegraded":"2024-09-21T01:20:42.170133+0000","last_fullsized":"2024-09-21T01:20:42.170133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:37:45.104535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834120+0000","last_change":"2024-09-21T01:19:06.717820+0000","last_active":"2024-09-21T01:21:40.834120+0000","last_peered":"2024-09-21T01:21:40.834120+0000","last_clean":"2024-09-21T01:21:40.834120+0000","last_became_active":"2024-09-21T01:19:06.717448+0000","last_became_peered":"2024-09-21T01:19:06.717448+0000","last_unstale":"2024-09-21T01:21:40.834120+0000","last_undegraded":"2024-09-21T01:21:40.834120+0000","last_fullsized":"2024-09-21T01:21:40.834120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:25.940418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.170465+0000","last_change":"2024-09-21T01:19:04.020940+0000","last_active":"2024-09-21T01:20:42.170465+0000","last_peered":"2024-09-21T01:20:42.170465+0000","last_clean":"2024-09-21T01:20:42.170465+0000","last_became_active":"2024-09-21T01:19:04.020433+0000","last_became_peered":"2024-09-21T01:19:04.020433+0000","last_unstale":"2024-09-21T01:20:42.170465+0000","last_undegraded":"2024-09-21T01:20:42.170465+0000","last_fullsized":"2024-09-21T01:20:42.170465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:52.820804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166302+0000","last_change":"2024-09-21T01:19:16.802712+0000","last_active":"2024-09-21T01:20:42.166302+0000","last_peered":"2024-09-21T01:20:42.166302+0000","last_clean":"2024-09-21T01:20:42.166302+0000","last_became_active":"2024-09-21T01:19:16.802256+0000","last_became_peered":"2024-09-21T01:19:16.802256+0000","last_unstale":"2024-09-21T01:20:42.166302+0000","last_undegraded":"2024-09-21T01:20:42.166302+0000","last_fullsized":"2024-09-21T01:20:42.166302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:32.551672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834414+0000","last_change":"2024-09-21T01:19:06.708290+0000","last_active":"2024-09-21T01:21:40.834414+0000","last_peered":"2024-09-21T01:21:40.834414+0000","last_clean":"2024-09-21T01:21:40.834414+0000","last_became_active":"2024-09-21T01:19:06.706996+0000","last_became_peered":"2024-09-21T01:19:06.706996+0000","last_unstale":"2024-09-21T01:21:40.834414+0000","last_undegraded":"2024-09-21T01:21:40.834414+0000","last_fullsized":"2024-09-21T01:21:40.834414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:22.730254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'103","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:19.469016+0000","last_change":"2024-09-21T01:19:03.684731+0000","last_active":"2024-09-21T01:22:19.469016+0000","last_peered":"2024-09-21T01:22:19.469016+0000","last_clean":"2024-09-21T01:22:19.469016+0000","last_became_active":"2024-09-21T01:19:03.684250+0000","last_became_peered":"2024-09-21T01:19:03.684250+0000","last_unstale":"2024-09-21T01:22:19.469016+0000","last_undegraded":"2024-09-21T01:22:19.469016+0000","last_fullsized":"2024-09-21T01:22:19.469016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:24.796870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":188,"num_write_kb":1223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212484+0000","last_change":"2024-09-21T01:19:17.248343+0000","last_active":"2024-09-21T01:20:42.212484+0000","last_peered":"2024-09-21T01:20:42.212484+0000","last_clean":"2024-09-21T01:20:42.212484+0000","last_became_active":"2024-09-21T01:19:17.247809+0000","last_became_peered":"2024-09-21T01:19:17.247809+0000","last_unstale":"2024-09-21T01:20:42.212484+0000","last_undegraded":"2024-09-21T01:20:42.212484+0000","last_fullsized":"2024-09-21T01:20:42.212484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:40:48.044344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.814422+0000","last_change":"2024-09-21T01:19:06.716558+0000","last_active":"2024-09-21T01:21:40.814422+0000","last_peered":"2024-09-21T01:21:40.814422+0000","last_clean":"2024-09-21T01:21:40.814422+0000","last_became_active":"2024-09-21T01:19:06.714144+0000","last_became_peered":"2024-09-21T01:19:06.714144+0000","last_unstale":"2024-09-21T01:21:40.814422+0000","last_undegraded":"2024-09-21T01:21:40.814422+0000","last_fullsized":"2024-09-21T01:21:40.814422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:10:09.834472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.185170+0000","last_change":"2024-09-21T01:20:42.184446+0000","last_active":"2024-09-21T01:20:42.185170+0000","last_peered":"2024-09-21T01:20:42.185170+0000","last_clean":"2024-09-21T01:20:42.185170+0000","last_became_active":"2024-09-21T01:20:42.172338+0000","last_became_peered":"2024-09-21T01:20:42.172338+0000","last_unstale":"2024-09-21T01:20:42.185170+0000","last_undegraded":"2024-09-21T01:20:42.185170+0000","last_fullsized":"2024-09-21T01:20:42.185170+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:14:12.384489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168852+0000","last_change":"2024-09-21T01:19:16.806794+0000","last_active":"2024-09-21T01:20:42.168852+0000","last_peered":"2024-09-21T01:20:42.168852+0000","last_clean":"2024-09-21T01:20:42.168852+0000","last_became_active":"2024-09-21T01:19:16.806325+0000","last_became_peered":"2024-09-21T01:19:16.806325+0000","last_unstale":"2024-09-21T01:20:42.168852+0000","last_undegraded":"2024-09-21T01:20:42.168852+0000","last_fullsized":"2024-09-21T01:20:42.168852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:54.327050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832976+0000","last_change":"2024-09-21T01:19:06.724389+0000","last_active":"2024-09-21T01:21:40.832976+0000","last_peered":"2024-09-21T01:21:40.832976+0000","last_clean":"2024-09-21T01:21:40.832976+0000","last_became_active":"2024-09-21T01:19:06.723938+0000","last_became_peered":"2024-09-21T01:19:06.723938+0000","last_unstale":"2024-09-21T01:21:40.832976+0000","last_undegraded":"2024-09-21T01:21:40.832976+0000","last_fullsized":"2024-09-21T01:21:40.832976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:04:29.900154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167193+0000","last_change":"2024-09-21T01:19:03.694514+0000","last_active":"2024-09-21T01:20:42.167193+0000","last_peered":"2024-09-21T01:20:42.167193+0000","last_clean":"2024-09-21T01:20:42.167193+0000","last_became_active":"2024-09-21T01:19:03.693571+0000","last_became_peered":"2024-09-21T01:19:03.693571+0000","last_unstale":"2024-09-21T01:20:42.167193+0000","last_undegraded":"2024-09-21T01:20:42.167193+0000","last_fullsized":"2024-09-21T01:20:42.167193+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:50.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171244+0000","last_change":"2024-09-21T01:19:17.248291+0000","last_active":"2024-09-21T01:20:42.171244+0000","last_peered":"2024-09-21T01:20:42.171244+0000","last_clean":"2024-09-21T01:20:42.171244+0000","last_became_active":"2024-09-21T01:19:17.247717+0000","last_became_peered":"2024-09-21T01:19:17.247717+0000","last_unstale":"2024-09-21T01:20:42.171244+0000","last_undegraded":"2024-09-21T01:20:42.171244+0000","last_fullsized":"2024-09-21T01:20:42.171244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:14.213920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829906+0000","last_change":"2024-09-21T01:19:41.898699+0000","last_active":"2024-09-21T01:21:40.829906+0000","last_peered":"2024-09-21T01:21:40.829906+0000","last_clean":"2024-09-21T01:21:40.829906+0000","last_became_active":"2024-09-21T01:19:41.898239+0000","last_became_peered":"2024-09-21T01:19:41.898239+0000","last_unstale":"2024-09-21T01:21:40.829906+0000","last_undegraded":"2024-09-21T01:21:40.829906+0000","last_fullsized":"2024-09-21T01:21:40.829906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:32:52.462680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":188,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.611456+0000","last_change":"2024-09-21T01:19:03.681689+0000","last_active":"2024-09-21T01:22:18.611456+0000","last_peered":"2024-09-21T01:22:18.611456+0000","last_clean":"2024-09-21T01:22:18.611456+0000","last_became_active":"2024-09-21T01:19:03.680611+0000","last_became_peered":"2024-09-21T01:19:03.680611+0000","last_unstale":"2024-09-21T01:22:18.611456+0000","last_undegraded":"2024-09-21T01:22:18.611456+0000","last_fullsized":"2024-09-21T01:22:18.611456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:23.079835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":137,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368522+0000","last_change":"2024-09-21T01:19:16.808016+0000","last_active":"2024-09-21T01:20:42.368522+0000","last_peered":"2024-09-21T01:20:42.368522+0000","last_clean":"2024-09-21T01:20:42.368522+0000","last_became_active":"2024-09-21T01:19:16.807562+0000","last_became_peered":"2024-09-21T01:19:16.807562+0000","last_unstale":"2024-09-21T01:20:42.368522+0000","last_undegraded":"2024-09-21T01:20:42.368522+0000","last_fullsized":"2024-09-21T01:20:42.368522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:24:46.841792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824000+0000","last_change":"2024-09-21T01:19:06.711800+0000","last_active":"2024-09-21T01:21:40.824000+0000","last_peered":"2024-09-21T01:21:40.824000+0000","last_clean":"2024-09-21T01:21:40.824000+0000","last_became_active":"2024-09-21T01:19:06.711091+0000","last_became_peered":"2024-09-21T01:19:06.711091+0000","last_unstale":"2024-09-21T01:21:40.824000+0000","last_undegraded":"2024-09-21T01:21:40.824000+0000","last_fullsized":"2024-09-21T01:21:40.824000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:19:49.492414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":192,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:22.616036+0000","last_change":"2024-09-21T01:19:03.686287+0000","last_active":"2024-09-21T01:22:22.616036+0000","last_peered":"2024-09-21T01:22:22.616036+0000","last_clean":"2024-09-21T01:22:22.616036+0000","last_became_active":"2024-09-21T01:19:03.685812+0000","last_became_peered":"2024-09-21T01:19:03.685812+0000","last_unstale":"2024-09-21T01:22:22.616036+0000","last_undegraded":"2024-09-21T01:22:22.616036+0000","last_fullsized":"2024-09-21T01:22:22.616036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:39.259121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368663+0000","last_change":"2024-09-21T01:19:17.248373+0000","last_active":"2024-09-21T01:20:42.368663+0000","last_peered":"2024-09-21T01:20:42.368663+0000","last_clean":"2024-09-21T01:20:42.368663+0000","last_became_active":"2024-09-21T01:19:17.247757+0000","last_became_peered":"2024-09-21T01:19:17.247757+0000","last_unstale":"2024-09-21T01:20:42.368663+0000","last_undegraded":"2024-09-21T01:20:42.368663+0000","last_fullsized":"2024-09-21T01:20:42.368663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:32:19.962868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.811679+0000","last_change":"2024-09-21T01:19:06.712752+0000","last_active":"2024-09-21T01:21:40.811679+0000","last_peered":"2024-09-21T01:21:40.811679+0000","last_clean":"2024-09-21T01:21:40.811679+0000","last_became_active":"2024-09-21T01:19:06.710160+0000","last_became_peered":"2024-09-21T01:19:06.710160+0000","last_unstale":"2024-09-21T01:21:40.811679+0000","last_undegraded":"2024-09-21T01:21:40.811679+0000","last_fullsized":"2024-09-21T01:21:40.811679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:50.286476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166486+0000","last_change":"2024-09-21T01:19:03.682603+0000","last_active":"2024-09-21T01:20:42.166486+0000","last_peered":"2024-09-21T01:20:42.166486+0000","last_clean":"2024-09-21T01:20:42.166486+0000","last_became_active":"2024-09-21T01:19:03.682199+0000","last_became_peered":"2024-09-21T01:19:03.682199+0000","last_unstale":"2024-09-21T01:20:42.166486+0000","last_undegraded":"2024-09-21T01:20:42.166486+0000","last_fullsized":"2024-09-21T01:20:42.166486+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:55:33.729097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171878+0000","last_change":"2024-09-21T01:19:42.179665+0000","last_active":"2024-09-21T01:20:42.171878+0000","last_peered":"2024-09-21T01:20:42.171878+0000","last_clean":"2024-09-21T01:20:42.171878+0000","last_became_active":"2024-09-21T01:19:42.179036+0000","last_became_peered":"2024-09-21T01:19:42.179036+0000","last_unstale":"2024-09-21T01:20:42.171878+0000","last_undegraded":"2024-09-21T01:20:42.171878+0000","last_fullsized":"2024-09-21T01:20:42.171878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:02.799080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837018+0000","last_change":"2024-09-21T01:19:41.970420+0000","last_active":"2024-09-21T01:21:40.837018+0000","last_peered":"2024-09-21T01:21:40.837018+0000","last_clean":"2024-09-21T01:21:40.837018+0000","last_became_active":"2024-09-21T01:19:41.970137+0000","last_became_peered":"2024-09-21T01:19:41.970137+0000","last_unstale":"2024-09-21T01:21:40.837018+0000","last_undegraded":"2024-09-21T01:21:40.837018+0000","last_fullsized":"2024-09-21T01:21:40.837018+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:44:22.460969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169791+0000","last_change":"2024-09-21T01:19:03.681947+0000","last_active":"2024-09-21T01:20:42.169791+0000","last_peered":"2024-09-21T01:20:42.169791+0000","last_clean":"2024-09-21T01:20:42.169791+0000","last_became_active":"2024-09-21T01:19:03.681518+0000","last_became_peered":"2024-09-21T01:19:03.681518+0000","last_unstale":"2024-09-21T01:20:42.169791+0000","last_undegraded":"2024-09-21T01:20:42.169791+0000","last_fullsized":"2024-09-21T01:20:42.169791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:23:22.679521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166664+0000","last_change":"2024-09-21T01:17:43.292571+0000","last_active":"2024-09-21T01:20:42.166664+0000","last_peered":"2024-09-21T01:20:42.166664+0000","last_clean":"2024-09-21T01:20:42.166664+0000","last_became_active":"2024-09-21T01:17:43.208745+0000","last_became_peered":"2024-09-21T01:17:43.208745+0000","last_unstale":"2024-09-21T01:20:42.166664+0000","last_undegraded":"2024-09-21T01:20:42.166664+0000","last_fullsized":"2024-09-21T01:20:42.166664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:13:53.563306+0000","last_clean_scrub_stamp":"2024-09-21T01:13:53.563306+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:25.290200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166645+0000","last_change":"2024-09-21T01:19:16.818246+0000","last_active":"2024-09-21T01:20:42.166645+0000","last_peered":"2024-09-21T01:20:42.166645+0000","last_clean":"2024-09-21T01:20:42.166645+0000","last_became_active":"2024-09-21T01:19:16.817790+0000","last_became_peered":"2024-09-21T01:19:16.817790+0000","last_unstale":"2024-09-21T01:20:42.166645+0000","last_undegraded":"2024-09-21T01:20:42.166645+0000","last_fullsized":"2024-09-21T01:20:42.166645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:27:21.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838028+0000","last_change":"2024-09-21T01:19:06.715088+0000","last_active":"2024-09-21T01:21:40.838028+0000","last_peered":"2024-09-21T01:21:40.838028+0000","last_clean":"2024-09-21T01:21:40.838028+0000","last_became_active":"2024-09-21T01:19:06.714240+0000","last_became_peered":"2024-09-21T01:19:06.714240+0000","last_unstale":"2024-09-21T01:21:40.838028+0000","last_undegraded":"2024-09-21T01:21:40.838028+0000","last_fullsized":"2024-09-21T01:21:40.838028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:08:14.057208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.348297+0000","last_change":"2024-09-21T01:20:42.188731+0000","last_active":"2024-09-21T01:22:18.348297+0000","last_peered":"2024-09-21T01:22:18.348297+0000","last_clean":"2024-09-21T01:22:18.348297+0000","last_became_active":"2024-09-21T01:20:42.172041+0000","last_became_peered":"2024-09-21T01:20:42.172041+0000","last_unstale":"2024-09-21T01:22:18.348297+0000","last_undegraded":"2024-09-21T01:22:18.348297+0000","last_fullsized":"2024-09-21T01:22:18.348297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:42.392074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173149+0000","last_change":"2024-09-21T01:19:16.809544+0000","last_active":"2024-09-21T01:20:42.173149+0000","last_peered":"2024-09-21T01:20:42.173149+0000","last_clean":"2024-09-21T01:20:42.173149+0000","last_became_active":"2024-09-21T01:19:16.809073+0000","last_became_peered":"2024-09-21T01:19:16.809073+0000","last_unstale":"2024-09-21T01:20:42.173149+0000","last_undegraded":"2024-09-21T01:20:42.173149+0000","last_fullsized":"2024-09-21T01:20:42.173149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:02:27.995949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831868+0000","last_change":"2024-09-21T01:19:06.718556+0000","last_active":"2024-09-21T01:21:40.831868+0000","last_peered":"2024-09-21T01:21:40.831868+0000","last_clean":"2024-09-21T01:21:40.831868+0000","last_became_active":"2024-09-21T01:19:06.718086+0000","last_became_peered":"2024-09-21T01:19:06.718086+0000","last_unstale":"2024-09-21T01:21:40.831868+0000","last_undegraded":"2024-09-21T01:21:40.831868+0000","last_fullsized":"2024-09-21T01:21:40.831868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:00.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'62","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.972597+0000","last_change":"2024-09-21T01:19:03.683354+0000","last_active":"2024-09-21T01:22:18.972597+0000","last_peered":"2024-09-21T01:22:18.972597+0000","last_clean":"2024-09-21T01:22:18.972597+0000","last_became_active":"2024-09-21T01:19:03.682800+0000","last_became_peered":"2024-09-21T01:19:03.682800+0000","last_unstale":"2024-09-21T01:22:18.972597+0000","last_undegraded":"2024-09-21T01:22:18.972597+0000","last_fullsized":"2024-09-21T01:22:18.972597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:52.014771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171335+0000","last_change":"2024-09-21T01:19:16.821503+0000","last_active":"2024-09-21T01:20:42.171335+0000","last_peered":"2024-09-21T01:20:42.171335+0000","last_clean":"2024-09-21T01:20:42.171335+0000","last_became_active":"2024-09-21T01:19:16.821014+0000","last_became_peered":"2024-09-21T01:19:16.821014+0000","last_unstale":"2024-09-21T01:20:42.171335+0000","last_undegraded":"2024-09-21T01:20:42.171335+0000","last_fullsized":"2024-09-21T01:20:42.171335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:29.688447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830380+0000","last_change":"2024-09-21T01:19:06.706498+0000","last_active":"2024-09-21T01:21:40.830380+0000","last_peered":"2024-09-21T01:21:40.830380+0000","last_clean":"2024-09-21T01:21:40.830380+0000","last_became_active":"2024-09-21T01:19:06.706058+0000","last_became_peered":"2024-09-21T01:19:06.706058+0000","last_unstale":"2024-09-21T01:21:40.830380+0000","last_undegraded":"2024-09-21T01:21:40.830380+0000","last_fullsized":"2024-09-21T01:21:40.830380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:01:13.646899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368686+0000","last_change":"2024-09-21T01:19:03.687148+0000","last_active":"2024-09-21T01:20:42.368686+0000","last_peered":"2024-09-21T01:20:42.368686+0000","last_clean":"2024-09-21T01:20:42.368686+0000","last_became_active":"2024-09-21T01:19:03.686687+0000","last_became_peered":"2024-09-21T01:19:03.686687+0000","last_unstale":"2024-09-21T01:20:42.368686+0000","last_undegraded":"2024-09-21T01:20:42.368686+0000","last_fullsized":"2024-09-21T01:20:42.368686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:16.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173156+0000","last_change":"2024-09-21T01:19:16.806161+0000","last_active":"2024-09-21T01:20:42.173156+0000","last_peered":"2024-09-21T01:20:42.173156+0000","last_clean":"2024-09-21T01:20:42.173156+0000","last_became_active":"2024-09-21T01:19:16.805614+0000","last_became_peered":"2024-09-21T01:19:16.805614+0000","last_unstale":"2024-09-21T01:20:42.173156+0000","last_undegraded":"2024-09-21T01:20:42.173156+0000","last_fullsized":"2024-09-21T01:20:42.173156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:39.725154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824620+0000","last_change":"2024-09-21T01:19:06.704347+0000","last_active":"2024-09-21T01:21:40.824620+0000","last_peered":"2024-09-21T01:21:40.824620+0000","last_clean":"2024-09-21T01:21:40.824620+0000","last_became_active":"2024-09-21T01:19:06.703865+0000","last_became_peered":"2024-09-21T01:19:06.703865+0000","last_unstale":"2024-09-21T01:21:40.824620+0000","last_undegraded":"2024-09-21T01:21:40.824620+0000","last_fullsized":"2024-09-21T01:21:40.824620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:51.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.189333+0000","last_change":"2024-09-21T01:20:42.188808+0000","last_active":"2024-09-21T01:20:42.189333+0000","last_peered":"2024-09-21T01:20:42.189333+0000","last_clean":"2024-09-21T01:20:42.189333+0000","last_became_active":"2024-09-21T01:20:42.175348+0000","last_became_peered":"2024-09-21T01:20:42.175348+0000","last_unstale":"2024-09-21T01:20:42.189333+0000","last_undegraded":"2024-09-21T01:20:42.189333+0000","last_fullsized":"2024-09-21T01:20:42.189333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:57:25.746569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173579+0000","last_change":"2024-09-21T01:19:16.807950+0000","last_active":"2024-09-21T01:20:42.173579+0000","last_peered":"2024-09-21T01:20:42.173579+0000","last_clean":"2024-09-21T01:20:42.173579+0000","last_became_active":"2024-09-21T01:19:16.807517+0000","last_became_peered":"2024-09-21T01:19:16.807517+0000","last_unstale":"2024-09-21T01:20:42.173579+0000","last_undegraded":"2024-09-21T01:20:42.173579+0000","last_fullsized":"2024-09-21T01:20:42.173579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:11:00.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.839227+0000","last_change":"2024-09-21T01:19:06.718450+0000","last_active":"2024-09-21T01:21:40.839227+0000","last_peered":"2024-09-21T01:21:40.839227+0000","last_clean":"2024-09-21T01:21:40.839227+0000","last_became_active":"2024-09-21T01:19:06.716541+0000","last_became_peered":"2024-09-21T01:19:06.716541+0000","last_unstale":"2024-09-21T01:21:40.839227+0000","last_undegraded":"2024-09-21T01:21:40.839227+0000","last_fullsized":"2024-09-21T01:21:40.839227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:58.300989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171236+0000","last_change":"2024-09-21T01:19:03.685349+0000","last_active":"2024-09-21T01:20:42.171236+0000","last_peered":"2024-09-21T01:20:42.171236+0000","last_clean":"2024-09-21T01:20:42.171236+0000","last_became_active":"2024-09-21T01:19:03.684941+0000","last_became_peered":"2024-09-21T01:19:03.684941+0000","last_unstale":"2024-09-21T01:20:42.171236+0000","last_undegraded":"2024-09-21T01:20:42.171236+0000","last_fullsized":"2024-09-21T01:20:42.171236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:03.852766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368646+0000","last_change":"2024-09-21T01:19:16.818507+0000","last_active":"2024-09-21T01:20:42.368646+0000","last_peered":"2024-09-21T01:20:42.368646+0000","last_clean":"2024-09-21T01:20:42.368646+0000","last_became_active":"2024-09-21T01:19:16.817962+0000","last_became_peered":"2024-09-21T01:19:16.817962+0000","last_unstale":"2024-09-21T01:20:42.368646+0000","last_undegraded":"2024-09-21T01:20:42.368646+0000","last_fullsized":"2024-09-21T01:20:42.368646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:37.568746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831667+0000","last_change":"2024-09-21T01:19:06.711294+0000","last_active":"2024-09-21T01:21:40.831667+0000","last_peered":"2024-09-21T01:21:40.831667+0000","last_clean":"2024-09-21T01:21:40.831667+0000","last_became_active":"2024-09-21T01:19:06.710900+0000","last_became_peered":"2024-09-21T01:19:06.710900+0000","last_unstale":"2024-09-21T01:21:40.831667+0000","last_undegraded":"2024-09-21T01:21:40.831667+0000","last_fullsized":"2024-09-21T01:21:40.831667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:49.968794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":191,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:18.610592+0000","last_change":"2024-09-21T01:19:03.681918+0000","last_active":"2024-09-21T01:22:18.610592+0000","last_peered":"2024-09-21T01:22:18.610592+0000","last_clean":"2024-09-21T01:22:18.610592+0000","last_became_active":"2024-09-21T01:19:03.681503+0000","last_became_peered":"2024-09-21T01:19:03.681503+0000","last_unstale":"2024-09-21T01:22:18.610592+0000","last_undegraded":"2024-09-21T01:22:18.610592+0000","last_fullsized":"2024-09-21T01:22:18.610592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:55.713146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.172600+0000","last_change":"2024-09-21T01:19:42.104405+0000","last_active":"2024-09-21T01:20:42.172600+0000","last_peered":"2024-09-21T01:20:42.172600+0000","last_clean":"2024-09-21T01:20:42.172600+0000","last_became_active":"2024-09-21T01:19:42.104033+0000","last_became_peered":"2024-09-21T01:19:42.104033+0000","last_unstale":"2024-09-21T01:20:42.172600+0000","last_undegraded":"2024-09-21T01:20:42.172600+0000","last_fullsized":"2024-09-21T01:20:42.172600+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:57.534937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832208+0000","last_change":"2024-09-21T01:19:06.709340+0000","last_active":"2024-09-21T01:21:40.832208+0000","last_peered":"2024-09-21T01:21:40.832208+0000","last_clean":"2024-09-21T01:21:40.832208+0000","last_became_active":"2024-09-21T01:19:06.708947+0000","last_became_peered":"2024-09-21T01:19:06.708947+0000","last_unstale":"2024-09-21T01:21:40.832208+0000","last_undegraded":"2024-09-21T01:21:40.832208+0000","last_fullsized":"2024-09-21T01:21:40.832208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:32.339854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173980+0000","last_change":"2024-09-21T01:19:03.682350+0000","last_active":"2024-09-21T01:20:42.173980+0000","last_peered":"2024-09-21T01:20:42.173980+0000","last_clean":"2024-09-21T01:20:42.173980+0000","last_became_active":"2024-09-21T01:19:03.680120+0000","last_became_peered":"2024-09-21T01:19:03.680120+0000","last_unstale":"2024-09-21T01:20:42.173980+0000","last_undegraded":"2024-09-21T01:20:42.173980+0000","last_fullsized":"2024-09-21T01:20:42.173980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:15.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.168815+0000","last_change":"2024-09-21T01:19:16.818787+0000","last_active":"2024-09-21T01:20:42.168815+0000","last_peered":"2024-09-21T01:20:42.168815+0000","last_clean":"2024-09-21T01:20:42.168815+0000","last_became_active":"2024-09-21T01:19:16.818267+0000","last_became_peered":"2024-09-21T01:19:16.818267+0000","last_unstale":"2024-09-21T01:20:42.168815+0000","last_undegraded":"2024-09-21T01:20:42.168815+0000","last_fullsized":"2024-09-21T01:20:42.168815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:41.673139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829263+0000","last_change":"2024-09-21T01:19:06.729996+0000","last_active":"2024-09-21T01:21:40.829263+0000","last_peered":"2024-09-21T01:21:40.829263+0000","last_clean":"2024-09-21T01:21:40.829263+0000","last_became_active":"2024-09-21T01:19:06.729621+0000","last_became_peered":"2024-09-21T01:19:06.729621+0000","last_unstale":"2024-09-21T01:21:40.829263+0000","last_undegraded":"2024-09-21T01:21:40.829263+0000","last_fullsized":"2024-09-21T01:21:40.829263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:29.669097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166698+0000","last_change":"2024-09-21T01:19:03.687472+0000","last_active":"2024-09-21T01:20:42.166698+0000","last_peered":"2024-09-21T01:20:42.166698+0000","last_clean":"2024-09-21T01:20:42.166698+0000","last_became_active":"2024-09-21T01:19:03.686702+0000","last_became_peered":"2024-09-21T01:19:03.686702+0000","last_unstale":"2024-09-21T01:20:42.166698+0000","last_undegraded":"2024-09-21T01:20:42.166698+0000","last_fullsized":"2024-09-21T01:20:42.166698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:09:58.594814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173990+0000","last_change":"2024-09-21T01:19:16.813938+0000","last_active":"2024-09-21T01:20:42.173990+0000","last_peered":"2024-09-21T01:20:42.173990+0000","last_clean":"2024-09-21T01:20:42.173990+0000","last_became_active":"2024-09-21T01:19:16.813404+0000","last_became_peered":"2024-09-21T01:19:16.813404+0000","last_unstale":"2024-09-21T01:20:42.173990+0000","last_undegraded":"2024-09-21T01:20:42.173990+0000","last_fullsized":"2024-09-21T01:20:42.173990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:21:42.666863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.810339+0000","last_change":"2024-09-21T01:19:06.714010+0000","last_active":"2024-09-21T01:21:40.810339+0000","last_peered":"2024-09-21T01:21:40.810339+0000","last_clean":"2024-09-21T01:21:40.810339+0000","last_became_active":"2024-09-21T01:19:06.713638+0000","last_became_peered":"2024-09-21T01:19:06.713638+0000","last_unstale":"2024-09-21T01:21:40.810339+0000","last_undegraded":"2024-09-21T01:21:40.810339+0000","last_fullsized":"2024-09-21T01:21:40.810339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:37.160311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":392,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:20.333237+0000","last_change":"2024-09-21T01:20:42.182111+0000","last_active":"2024-09-21T01:22:20.333237+0000","last_peered":"2024-09-21T01:22:20.333237+0000","last_clean":"2024-09-21T01:22:20.333237+0000","last_became_active":"2024-09-21T01:20:42.174484+0000","last_became_peered":"2024-09-21T01:20:42.174484+0000","last_unstale":"2024-09-21T01:22:20.333237+0000","last_undegraded":"2024-09-21T01:22:20.333237+0000","last_fullsized":"2024-09-21T01:22:20.333237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:45.448208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75126,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":281,"num_read_kb":214,"num_write":44,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":66212,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166872+0000","last_change":"2024-09-21T01:19:16.814662+0000","last_active":"2024-09-21T01:20:42.166872+0000","last_peered":"2024-09-21T01:20:42.166872+0000","last_clean":"2024-09-21T01:20:42.166872+0000","last_became_active":"2024-09-21T01:19:16.814225+0000","last_became_peered":"2024-09-21T01:19:16.814225+0000","last_unstale":"2024-09-21T01:20:42.166872+0000","last_undegraded":"2024-09-21T01:20:42.166872+0000","last_fullsized":"2024-09-21T01:20:42.166872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:51.778985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838943+0000","last_change":"2024-09-21T01:19:06.713250+0000","last_active":"2024-09-21T01:21:40.838943+0000","last_peered":"2024-09-21T01:21:40.838943+0000","last_clean":"2024-09-21T01:21:40.838943+0000","last_became_active":"2024-09-21T01:19:06.712850+0000","last_became_peered":"2024-09-21T01:19:06.712850+0000","last_unstale":"2024-09-21T01:21:40.838943+0000","last_undegraded":"2024-09-21T01:21:40.838943+0000","last_fullsized":"2024-09-21T01:21:40.838943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:57:08.839907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.174291+0000","last_change":"2024-09-21T01:19:03.688813+0000","last_active":"2024-09-21T01:20:42.174291+0000","last_peered":"2024-09-21T01:20:42.174291+0000","last_clean":"2024-09-21T01:20:42.174291+0000","last_became_active":"2024-09-21T01:19:03.688339+0000","last_became_peered":"2024-09-21T01:19:03.688339+0000","last_unstale":"2024-09-21T01:20:42.174291+0000","last_undegraded":"2024-09-21T01:20:42.174291+0000","last_fullsized":"2024-09-21T01:20:42.174291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:38:48.289609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169605+0000","last_change":"2024-09-21T01:19:16.821217+0000","last_active":"2024-09-21T01:20:42.169605+0000","last_peered":"2024-09-21T01:20:42.169605+0000","last_clean":"2024-09-21T01:20:42.169605+0000","last_became_active":"2024-09-21T01:19:16.820435+0000","last_became_peered":"2024-09-21T01:19:16.820435+0000","last_unstale":"2024-09-21T01:20:42.169605+0000","last_undegraded":"2024-09-21T01:20:42.169605+0000","last_fullsized":"2024-09-21T01:20:42.169605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:03.380569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.822722+0000","last_change":"2024-09-21T01:19:06.720491+0000","last_active":"2024-09-21T01:21:40.822722+0000","last_peered":"2024-09-21T01:21:40.822722+0000","last_clean":"2024-09-21T01:21:40.822722+0000","last_became_active":"2024-09-21T01:19:06.719791+0000","last_became_peered":"2024-09-21T01:19:06.719791+0000","last_unstale":"2024-09-21T01:21:40.822722+0000","last_undegraded":"2024-09-21T01:21:40.822722+0000","last_fullsized":"2024-09-21T01:21:40.822722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:48:40.997910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.167148+0000","last_change":"2024-09-21T01:19:03.682529+0000","last_active":"2024-09-21T01:20:42.167148+0000","last_peered":"2024-09-21T01:20:42.167148+0000","last_clean":"2024-09-21T01:20:42.167148+0000","last_became_active":"2024-09-21T01:19:03.682075+0000","last_became_peered":"2024-09-21T01:19:03.682075+0000","last_unstale":"2024-09-21T01:20:42.167148+0000","last_undegraded":"2024-09-21T01:20:42.167148+0000","last_fullsized":"2024-09-21T01:20:42.167148+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:34:43.980778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166594+0000","last_change":"2024-09-21T01:19:16.816540+0000","last_active":"2024-09-21T01:20:42.166594+0000","last_peered":"2024-09-21T01:20:42.166594+0000","last_clean":"2024-09-21T01:20:42.166594+0000","last_became_active":"2024-09-21T01:19:16.813925+0000","last_became_peered":"2024-09-21T01:19:16.813925+0000","last_unstale":"2024-09-21T01:20:42.166594+0000","last_undegraded":"2024-09-21T01:20:42.166594+0000","last_fullsized":"2024-09-21T01:20:42.166594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:51.610698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837693+0000","last_change":"2024-09-21T01:19:06.710229+0000","last_active":"2024-09-21T01:21:40.837693+0000","last_peered":"2024-09-21T01:21:40.837693+0000","last_clean":"2024-09-21T01:21:40.837693+0000","last_became_active":"2024-09-21T01:19:06.709808+0000","last_became_peered":"2024-09-21T01:19:06.709808+0000","last_unstale":"2024-09-21T01:21:40.837693+0000","last_undegraded":"2024-09-21T01:21:40.837693+0000","last_fullsized":"2024-09-21T01:21:40.837693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:21.993147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":322,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:19.612424+0000","last_change":"2024-09-21T01:19:03.680596+0000","last_active":"2024-09-21T01:22:19.612424+0000","last_peered":"2024-09-21T01:22:19.612424+0000","last_clean":"2024-09-21T01:22:19.612424+0000","last_became_active":"2024-09-21T01:19:03.680210+0000","last_became_peered":"2024-09-21T01:19:03.680210+0000","last_unstale":"2024-09-21T01:22:19.612424+0000","last_undegraded":"2024-09-21T01:22:19.612424+0000","last_fullsized":"2024-09-21T01:22:19.612424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:46:02.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.212589+0000","last_change":"2024-09-21T01:19:16.809177+0000","last_active":"2024-09-21T01:20:42.212589+0000","last_peered":"2024-09-21T01:20:42.212589+0000","last_clean":"2024-09-21T01:20:42.212589+0000","last_became_active":"2024-09-21T01:19:16.808648+0000","last_became_peered":"2024-09-21T01:19:16.808648+0000","last_unstale":"2024-09-21T01:20:42.212589+0000","last_undegraded":"2024-09-21T01:20:42.212589+0000","last_fullsized":"2024-09-21T01:20:42.212589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:12.447010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.171758+0000","last_change":"2024-09-21T01:19:16.810386+0000","last_active":"2024-09-21T01:20:42.171758+0000","last_peered":"2024-09-21T01:20:42.171758+0000","last_clean":"2024-09-21T01:20:42.171758+0000","last_became_active":"2024-09-21T01:19:16.809245+0000","last_became_peered":"2024-09-21T01:19:16.809245+0000","last_unstale":"2024-09-21T01:20:42.171758+0000","last_undegraded":"2024-09-21T01:20:42.171758+0000","last_fullsized":"2024-09-21T01:20:42.171758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:31.724729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.827678+0000","last_change":"2024-09-21T01:19:06.711479+0000","last_active":"2024-09-21T01:21:40.827678+0000","last_peered":"2024-09-21T01:21:40.827678+0000","last_clean":"2024-09-21T01:21:40.827678+0000","last_became_active":"2024-09-21T01:19:06.710042+0000","last_became_peered":"2024-09-21T01:19:06.710042+0000","last_unstale":"2024-09-21T01:21:40.827678+0000","last_undegraded":"2024-09-21T01:21:40.827678+0000","last_fullsized":"2024-09-21T01:21:40.827678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:52.781167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.173442+0000","last_change":"2024-09-21T01:19:03.682543+0000","last_active":"2024-09-21T01:20:42.173442+0000","last_peered":"2024-09-21T01:20:42.173442+0000","last_clean":"2024-09-21T01:20:42.173442+0000","last_became_active":"2024-09-21T01:19:03.682148+0000","last_became_peered":"2024-09-21T01:19:03.682148+0000","last_unstale":"2024-09-21T01:20:42.173442+0000","last_undegraded":"2024-09-21T01:20:42.173442+0000","last_fullsized":"2024-09-21T01:20:42.173442+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:03.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169158+0000","last_change":"2024-09-21T01:19:41.899698+0000","last_active":"2024-09-21T01:20:42.169158+0000","last_peered":"2024-09-21T01:20:42.169158+0000","last_clean":"2024-09-21T01:20:42.169158+0000","last_became_active":"2024-09-21T01:19:41.899115+0000","last_became_peered":"2024-09-21T01:19:41.899115+0000","last_unstale":"2024-09-21T01:20:42.169158+0000","last_undegraded":"2024-09-21T01:20:42.169158+0000","last_fullsized":"2024-09-21T01:20:42.169158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:35.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833231+0000","last_change":"2024-09-21T01:19:06.705566+0000","last_active":"2024-09-21T01:21:40.833231+0000","last_peered":"2024-09-21T01:21:40.833231+0000","last_clean":"2024-09-21T01:21:40.833231+0000","last_became_active":"2024-09-21T01:19:06.705158+0000","last_became_peered":"2024-09-21T01:19:06.705158+0000","last_unstale":"2024-09-21T01:21:40.833231+0000","last_undegraded":"2024-09-21T01:21:40.833231+0000","last_fullsized":"2024-09-21T01:21:40.833231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:34.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.368185+0000","last_change":"2024-09-21T01:19:03.682088+0000","last_active":"2024-09-21T01:20:42.368185+0000","last_peered":"2024-09-21T01:20:42.368185+0000","last_clean":"2024-09-21T01:20:42.368185+0000","last_became_active":"2024-09-21T01:19:03.681322+0000","last_became_peered":"2024-09-21T01:19:03.681322+0000","last_unstale":"2024-09-21T01:20:42.368185+0000","last_undegraded":"2024-09-21T01:20:42.368185+0000","last_fullsized":"2024-09-21T01:20:42.368185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:19.235217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.169957+0000","last_change":"2024-09-21T01:19:03.686984+0000","last_active":"2024-09-21T01:20:42.169957+0000","last_peered":"2024-09-21T01:20:42.169957+0000","last_clean":"2024-09-21T01:20:42.169957+0000","last_became_active":"2024-09-21T01:19:03.686568+0000","last_became_peered":"2024-09-21T01:19:03.686568+0000","last_unstale":"2024-09-21T01:20:42.169957+0000","last_undegraded":"2024-09-21T01:20:42.169957+0000","last_fullsized":"2024-09-21T01:20:42.169957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:52.621177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833060+0000","last_change":"2024-09-21T01:19:06.703643+0000","last_active":"2024-09-21T01:21:40.833060+0000","last_peered":"2024-09-21T01:21:40.833060+0000","last_clean":"2024-09-21T01:21:40.833060+0000","last_became_active":"2024-09-21T01:19:06.703225+0000","last_became_peered":"2024-09-21T01:19:06.703225+0000","last_unstale":"2024-09-21T01:21:40.833060+0000","last_undegraded":"2024-09-21T01:21:40.833060+0000","last_fullsized":"2024-09-21T01:21:40.833060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:12.817482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166300+0000","last_change":"2024-09-21T01:19:16.822226+0000","last_active":"2024-09-21T01:20:42.166300+0000","last_peered":"2024-09-21T01:20:42.166300+0000","last_clean":"2024-09-21T01:20:42.166300+0000","last_became_active":"2024-09-21T01:19:16.821821+0000","last_became_peered":"2024-09-21T01:19:16.821821+0000","last_unstale":"2024-09-21T01:20:42.166300+0000","last_undegraded":"2024-09-21T01:20:42.166300+0000","last_fullsized":"2024-09-21T01:20:42.166300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:13:30.197838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":195,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:22:20.614791+0000","last_change":"2024-09-21T01:19:03.682373+0000","last_active":"2024-09-21T01:22:20.614791+0000","last_peered":"2024-09-21T01:22:20.614791+0000","last_clean":"2024-09-21T01:22:20.614791+0000","last_became_active":"2024-09-21T01:19:03.681368+0000","last_became_peered":"2024-09-21T01:19:03.681368+0000","last_unstale":"2024-09-21T01:22:20.614791+0000","last_undegraded":"2024-09-21T01:22:20.614791+0000","last_fullsized":"2024-09-21T01:22:20.614791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:02.654574+0000","last_clean_scrub_stamp":"2024-09-21T01:19:02.654574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.939021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1465,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.812341+0000","last_change":"2024-09-21T01:19:06.709599+0000","last_active":"2024-09-21T01:21:40.812341+0000","last_peered":"2024-09-21T01:21:40.812341+0000","last_clean":"2024-09-21T01:21:40.812341+0000","last_became_active":"2024-09-21T01:19:06.709232+0000","last_became_peered":"2024-09-21T01:19:06.709232+0000","last_unstale":"2024-09-21T01:21:40.812341+0000","last_undegraded":"2024-09-21T01:21:40.812341+0000","last_fullsized":"2024-09-21T01:21:40.812341+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:09:12.953414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:20:42.166566+0000","last_change":"2024-09-21T01:19:16.806805+0000","last_active":"2024-09-21T01:20:42.166566+0000","last_peered":"2024-09-21T01:20:42.166566+0000","last_clean":"2024-09-21T01:20:42.166566+0000","last_became_active":"2024-09-21T01:19:16.806294+0000","last_became_peered":"2024-09-21T01:19:16.806294+0000","last_unstale":"2024-09-21T01:20:42.166566+0000","last_undegraded":"2024-09-21T01:20:42.166566+0000","last_fullsized":"2024-09-21T01:20:42.166566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:15.773859+0000","last_clean_scrub_stamp":"2024-09-21T01:19:15.773859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:50:55.623382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.825083+0000","last_change":"2024-09-21T01:19:06.708175+0000","last_active":"2024-09-21T01:21:40.825083+0000","last_peered":"2024-09-21T01:21:40.825083+0000","last_clean":"2024-09-21T01:21:40.825083+0000","last_became_active":"2024-09-21T01:19:06.707699+0000","last_became_peered":"2024-09-21T01:19:06.707699+0000","last_unstale":"2024-09-21T01:21:40.825083+0000","last_undegraded":"2024-09-21T01:21:40.825083+0000","last_fullsized":"2024-09-21T01:21:40.825083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:23:38.845784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.823972+0000","last_change":"2024-09-21T01:19:41.902192+0000","last_active":"2024-09-21T01:21:40.823972+0000","last_peered":"2024-09-21T01:21:40.823972+0000","last_clean":"2024-09-21T01:21:40.823972+0000","last_became_active":"2024-09-21T01:19:41.901723+0000","last_became_peered":"2024-09-21T01:19:41.901723+0000","last_unstale":"2024-09-21T01:21:40.823972+0000","last_undegraded":"2024-09-21T01:21:40.823972+0000","last_fullsized":"2024-09-21T01:21:40.823972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:41.713392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831722+0000","last_change":"2024-09-21T01:19:06.726634+0000","last_active":"2024-09-21T01:21:40.831722+0000","last_peered":"2024-09-21T01:21:40.831722+0000","last_clean":"2024-09-21T01:21:40.831722+0000","last_became_active":"2024-09-21T01:19:06.726321+0000","last_became_peered":"2024-09-21T01:19:06.726321+0000","last_unstale":"2024-09-21T01:21:40.831722+0000","last_undegraded":"2024-09-21T01:21:40.831722+0000","last_fullsized":"2024-09-21T01:21:40.831722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:46.736969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830027+0000","last_change":"2024-09-21T01:19:06.732631+0000","last_active":"2024-09-21T01:21:40.830027+0000","last_peered":"2024-09-21T01:21:40.830027+0000","last_clean":"2024-09-21T01:21:40.830027+0000","last_became_active":"2024-09-21T01:19:06.732191+0000","last_became_peered":"2024-09-21T01:19:06.732191+0000","last_unstale":"2024-09-21T01:21:40.830027+0000","last_undegraded":"2024-09-21T01:21:40.830027+0000","last_fullsized":"2024-09-21T01:21:40.830027+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:54:30.899851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829116+0000","last_change":"2024-09-21T01:19:06.719305+0000","last_active":"2024-09-21T01:21:40.829116+0000","last_peered":"2024-09-21T01:21:40.829116+0000","last_clean":"2024-09-21T01:21:40.829116+0000","last_became_active":"2024-09-21T01:19:06.714649+0000","last_became_peered":"2024-09-21T01:19:06.714649+0000","last_unstale":"2024-09-21T01:21:40.829116+0000","last_undegraded":"2024-09-21T01:21:40.829116+0000","last_fullsized":"2024-09-21T01:21:40.829116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:28:34.430275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.824234+0000","last_change":"2024-09-21T01:19:06.711854+0000","last_active":"2024-09-21T01:21:40.824234+0000","last_peered":"2024-09-21T01:21:40.824234+0000","last_clean":"2024-09-21T01:21:40.824234+0000","last_became_active":"2024-09-21T01:19:06.711495+0000","last_became_peered":"2024-09-21T01:19:06.711495+0000","last_unstale":"2024-09-21T01:21:40.824234+0000","last_undegraded":"2024-09-21T01:21:40.824234+0000","last_fullsized":"2024-09-21T01:21:40.824234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:17.980480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.831569+0000","last_change":"2024-09-21T01:19:06.720202+0000","last_active":"2024-09-21T01:21:40.831569+0000","last_peered":"2024-09-21T01:21:40.831569+0000","last_clean":"2024-09-21T01:21:40.831569+0000","last_became_active":"2024-09-21T01:19:06.719819+0000","last_became_peered":"2024-09-21T01:19:06.719819+0000","last_unstale":"2024-09-21T01:21:40.831569+0000","last_undegraded":"2024-09-21T01:21:40.831569+0000","last_fullsized":"2024-09-21T01:21:40.831569+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.112357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832623+0000","last_change":"2024-09-21T01:19:06.724326+0000","last_active":"2024-09-21T01:21:40.832623+0000","last_peered":"2024-09-21T01:21:40.832623+0000","last_clean":"2024-09-21T01:21:40.832623+0000","last_became_active":"2024-09-21T01:19:06.723883+0000","last_became_peered":"2024-09-21T01:19:06.723883+0000","last_unstale":"2024-09-21T01:21:40.832623+0000","last_undegraded":"2024-09-21T01:21:40.832623+0000","last_fullsized":"2024-09-21T01:21:40.832623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:28.844854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.832822+0000","last_change":"2024-09-21T01:19:06.708076+0000","last_active":"2024-09-21T01:21:40.832822+0000","last_peered":"2024-09-21T01:21:40.832822+0000","last_clean":"2024-09-21T01:21:40.832822+0000","last_became_active":"2024-09-21T01:19:06.707722+0000","last_became_peered":"2024-09-21T01:19:06.707722+0000","last_unstale":"2024-09-21T01:21:40.832822+0000","last_undegraded":"2024-09-21T01:21:40.832822+0000","last_fullsized":"2024-09-21T01:21:40.832822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:48:59.303512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.833468+0000","last_change":"2024-09-21T01:19:06.723304+0000","last_active":"2024-09-21T01:21:40.833468+0000","last_peered":"2024-09-21T01:21:40.833468+0000","last_clean":"2024-09-21T01:21:40.833468+0000","last_became_active":"2024-09-21T01:19:06.722973+0000","last_became_peered":"2024-09-21T01:19:06.722973+0000","last_unstale":"2024-09-21T01:21:40.833468+0000","last_undegraded":"2024-09-21T01:21:40.833468+0000","last_fullsized":"2024-09-21T01:21:40.833468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:45:51.709739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838351+0000","last_change":"2024-09-21T01:19:06.718042+0000","last_active":"2024-09-21T01:21:40.838351+0000","last_peered":"2024-09-21T01:21:40.838351+0000","last_clean":"2024-09-21T01:21:40.838351+0000","last_became_active":"2024-09-21T01:19:06.717627+0000","last_became_peered":"2024-09-21T01:19:06.717627+0000","last_unstale":"2024-09-21T01:21:40.838351+0000","last_undegraded":"2024-09-21T01:21:40.838351+0000","last_fullsized":"2024-09-21T01:21:40.838351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:47:55.575758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830634+0000","last_change":"2024-09-21T01:19:06.726266+0000","last_active":"2024-09-21T01:21:40.830634+0000","last_peered":"2024-09-21T01:21:40.830634+0000","last_clean":"2024-09-21T01:21:40.830634+0000","last_became_active":"2024-09-21T01:19:06.720039+0000","last_became_peered":"2024-09-21T01:19:06.720039+0000","last_unstale":"2024-09-21T01:21:40.830634+0000","last_undegraded":"2024-09-21T01:21:40.830634+0000","last_fullsized":"2024-09-21T01:21:40.830634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:35:43.530990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.823049+0000","last_change":"2024-09-21T01:19:06.710535+0000","last_active":"2024-09-21T01:21:40.823049+0000","last_peered":"2024-09-21T01:21:40.823049+0000","last_clean":"2024-09-21T01:21:40.823049+0000","last_became_active":"2024-09-21T01:19:06.710134+0000","last_became_peered":"2024-09-21T01:19:06.710134+0000","last_unstale":"2024-09-21T01:21:40.823049+0000","last_undegraded":"2024-09-21T01:21:40.823049+0000","last_fullsized":"2024-09-21T01:21:40.823049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:43:36.509712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834378+0000","last_change":"2024-09-21T01:19:06.731085+0000","last_active":"2024-09-21T01:21:40.834378+0000","last_peered":"2024-09-21T01:21:40.834378+0000","last_clean":"2024-09-21T01:21:40.834378+0000","last_became_active":"2024-09-21T01:19:06.730755+0000","last_became_peered":"2024-09-21T01:19:06.730755+0000","last_unstale":"2024-09-21T01:21:40.834378+0000","last_undegraded":"2024-09-21T01:21:40.834378+0000","last_fullsized":"2024-09-21T01:21:40.834378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:56:15.815060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837211+0000","last_change":"2024-09-21T01:19:06.710049+0000","last_active":"2024-09-21T01:21:40.837211+0000","last_peered":"2024-09-21T01:21:40.837211+0000","last_clean":"2024-09-21T01:21:40.837211+0000","last_became_active":"2024-09-21T01:19:06.709665+0000","last_became_peered":"2024-09-21T01:19:06.709665+0000","last_unstale":"2024-09-21T01:21:40.837211+0000","last_undegraded":"2024-09-21T01:21:40.837211+0000","last_fullsized":"2024-09-21T01:21:40.837211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:26.769249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830574+0000","last_change":"2024-09-21T01:19:06.713630+0000","last_active":"2024-09-21T01:21:40.830574+0000","last_peered":"2024-09-21T01:21:40.830574+0000","last_clean":"2024-09-21T01:21:40.830574+0000","last_became_active":"2024-09-21T01:19:06.713267+0000","last_became_peered":"2024-09-21T01:19:06.713267+0000","last_unstale":"2024-09-21T01:21:40.830574+0000","last_undegraded":"2024-09-21T01:21:40.830574+0000","last_fullsized":"2024-09-21T01:21:40.830574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:51:22.505322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.835525+0000","last_change":"2024-09-21T01:19:06.712357+0000","last_active":"2024-09-21T01:21:40.835525+0000","last_peered":"2024-09-21T01:21:40.835525+0000","last_clean":"2024-09-21T01:21:40.835525+0000","last_became_active":"2024-09-21T01:19:06.711993+0000","last_became_peered":"2024-09-21T01:19:06.711993+0000","last_unstale":"2024-09-21T01:21:40.835525+0000","last_undegraded":"2024-09-21T01:21:40.835525+0000","last_fullsized":"2024-09-21T01:21:40.835525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:52.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.828196+0000","last_change":"2024-09-21T01:19:06.718969+0000","last_active":"2024-09-21T01:21:40.828196+0000","last_peered":"2024-09-21T01:21:40.828196+0000","last_clean":"2024-09-21T01:21:40.828196+0000","last_became_active":"2024-09-21T01:19:06.718649+0000","last_became_peered":"2024-09-21T01:19:06.718649+0000","last_unstale":"2024-09-21T01:21:40.828196+0000","last_undegraded":"2024-09-21T01:21:40.828196+0000","last_fullsized":"2024-09-21T01:21:40.828196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:14.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.827534+0000","last_change":"2024-09-21T01:19:06.722111+0000","last_active":"2024-09-21T01:21:40.827534+0000","last_peered":"2024-09-21T01:21:40.827534+0000","last_clean":"2024-09-21T01:21:40.827534+0000","last_became_active":"2024-09-21T01:19:06.721748+0000","last_became_peered":"2024-09-21T01:19:06.721748+0000","last_unstale":"2024-09-21T01:21:40.827534+0000","last_undegraded":"2024-09-21T01:21:40.827534+0000","last_fullsized":"2024-09-21T01:21:40.827534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:53:42.217217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.829445+0000","last_change":"2024-09-21T01:19:06.726153+0000","last_active":"2024-09-21T01:21:40.829445+0000","last_peered":"2024-09-21T01:21:40.829445+0000","last_clean":"2024-09-21T01:21:40.829445+0000","last_became_active":"2024-09-21T01:19:06.719507+0000","last_became_peered":"2024-09-21T01:19:06.719507+0000","last_unstale":"2024-09-21T01:21:40.829445+0000","last_undegraded":"2024-09-21T01:21:40.829445+0000","last_fullsized":"2024-09-21T01:21:40.829445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:43.361215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.810535+0000","last_change":"2024-09-21T01:19:06.709191+0000","last_active":"2024-09-21T01:21:40.810535+0000","last_peered":"2024-09-21T01:21:40.810535+0000","last_clean":"2024-09-21T01:21:40.810535+0000","last_became_active":"2024-09-21T01:19:06.708695+0000","last_became_peered":"2024-09-21T01:19:06.708695+0000","last_unstale":"2024-09-21T01:21:40.810535+0000","last_undegraded":"2024-09-21T01:21:40.810535+0000","last_fullsized":"2024-09-21T01:21:40.810535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:27.397555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.838099+0000","last_change":"2024-09-21T01:19:06.715470+0000","last_active":"2024-09-21T01:21:40.838099+0000","last_peered":"2024-09-21T01:21:40.838099+0000","last_clean":"2024-09-21T01:21:40.838099+0000","last_became_active":"2024-09-21T01:19:06.714733+0000","last_became_peered":"2024-09-21T01:19:06.714733+0000","last_unstale":"2024-09-21T01:21:40.838099+0000","last_undegraded":"2024-09-21T01:21:40.838099+0000","last_fullsized":"2024-09-21T01:21:40.838099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:59.181589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.825971+0000","last_change":"2024-09-21T01:19:06.714497+0000","last_active":"2024-09-21T01:21:40.825971+0000","last_peered":"2024-09-21T01:21:40.825971+0000","last_clean":"2024-09-21T01:21:40.825971+0000","last_became_active":"2024-09-21T01:19:06.714078+0000","last_became_peered":"2024-09-21T01:19:06.714078+0000","last_unstale":"2024-09-21T01:21:40.825971+0000","last_undegraded":"2024-09-21T01:21:40.825971+0000","last_fullsized":"2024-09-21T01:21:40.825971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:45:09.653865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.837719+0000","last_change":"2024-09-21T01:19:06.718008+0000","last_active":"2024-09-21T01:21:40.837719+0000","last_peered":"2024-09-21T01:21:40.837719+0000","last_clean":"2024-09-21T01:21:40.837719+0000","last_became_active":"2024-09-21T01:19:06.717652+0000","last_became_peered":"2024-09-21T01:19:06.717652+0000","last_unstale":"2024-09-21T01:21:40.837719+0000","last_undegraded":"2024-09-21T01:21:40.837719+0000","last_fullsized":"2024-09-21T01:21:40.837719+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T02:37:17.013158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.834644+0000","last_change":"2024-09-21T01:19:06.718757+0000","last_active":"2024-09-21T01:21:40.834644+0000","last_peered":"2024-09-21T01:21:40.834644+0000","last_clean":"2024-09-21T01:21:40.834644+0000","last_became_active":"2024-09-21T01:19:06.718409+0000","last_became_peered":"2024-09-21T01:19:06.718409+0000","last_unstale":"2024-09-21T01:21:40.834644+0000","last_undegraded":"2024-09-21T01:21:40.834644+0000","last_fullsized":"2024-09-21T01:21:40.834644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:49.002534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830668+0000","last_change":"2024-09-21T01:19:06.711622+0000","last_active":"2024-09-21T01:21:40.830668+0000","last_peered":"2024-09-21T01:21:40.830668+0000","last_clean":"2024-09-21T01:21:40.830668+0000","last_became_active":"2024-09-21T01:19:06.711240+0000","last_became_peered":"2024-09-21T01:19:06.711240+0000","last_unstale":"2024-09-21T01:21:40.830668+0000","last_undegraded":"2024-09-21T01:21:40.830668+0000","last_fullsized":"2024-09-21T01:21:40.830668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:02:35.528841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.826190+0000","last_change":"2024-09-21T01:19:06.720907+0000","last_active":"2024-09-21T01:21:40.826190+0000","last_peered":"2024-09-21T01:21:40.826190+0000","last_clean":"2024-09-21T01:21:40.826190+0000","last_became_active":"2024-09-21T01:19:06.719941+0000","last_became_peered":"2024-09-21T01:19:06.719941+0000","last_unstale":"2024-09-21T01:21:40.826190+0000","last_undegraded":"2024-09-21T01:21:40.826190+0000","last_fullsized":"2024-09-21T01:21:40.826190+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:41.144633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.821398+0000","last_change":"2024-09-21T01:19:06.710898+0000","last_active":"2024-09-21T01:21:40.821398+0000","last_peered":"2024-09-21T01:21:40.821398+0000","last_clean":"2024-09-21T01:21:40.821398+0000","last_became_active":"2024-09-21T01:19:06.710492+0000","last_became_peered":"2024-09-21T01:19:06.710492+0000","last_unstale":"2024-09-21T01:21:40.821398+0000","last_undegraded":"2024-09-21T01:21:40.821398+0000","last_fullsized":"2024-09-21T01:21:40.821398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:08.559845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.817572+0000","last_change":"2024-09-21T01:19:06.710874+0000","last_active":"2024-09-21T01:21:40.817572+0000","last_peered":"2024-09-21T01:21:40.817572+0000","last_clean":"2024-09-21T01:21:40.817572+0000","last_became_active":"2024-09-21T01:19:06.710073+0000","last_became_peered":"2024-09-21T01:19:06.710073+0000","last_unstale":"2024-09-21T01:21:40.817572+0000","last_undegraded":"2024-09-21T01:21:40.817572+0000","last_fullsized":"2024-09-21T01:21:40.817572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:44.327067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830525+0000","last_change":"2024-09-21T01:19:06.715287+0000","last_active":"2024-09-21T01:21:40.830525+0000","last_peered":"2024-09-21T01:21:40.830525+0000","last_clean":"2024-09-21T01:21:40.830525+0000","last_became_active":"2024-09-21T01:19:06.714623+0000","last_became_peered":"2024-09-21T01:19:06.714623+0000","last_unstale":"2024-09-21T01:21:40.830525+0000","last_undegraded":"2024-09-21T01:21:40.830525+0000","last_fullsized":"2024-09-21T01:21:40.830525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T01:38:50.040768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.803325+0000","last_change":"2024-09-21T01:19:06.723902+0000","last_active":"2024-09-21T01:21:40.803325+0000","last_peered":"2024-09-21T01:21:40.803325+0000","last_clean":"2024-09-21T01:21:40.803325+0000","last_became_active":"2024-09-21T01:19:06.723489+0000","last_became_peered":"2024-09-21T01:19:06.723489+0000","last_unstale":"2024-09-21T01:21:40.803325+0000","last_undegraded":"2024-09-21T01:21:40.803325+0000","last_fullsized":"2024-09-21T01:21:40.803325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:16:48.062507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T01:21:40.830302+0000","last_change":"2024-09-21T01:19:06.717709+0000","last_active":"2024-09-21T01:21:40.830302+0000","last_peered":"2024-09-21T01:21:40.830302+0000","last_clean":"2024-09-21T01:21:40.830302+0000","last_became_active":"2024-09-21T01:19:06.717208+0000","last_became_peered":"2024-09-21T01:19:06.717208+0000","last_unstale":"2024-09-21T01:21:40.830302+0000","last_undegraded":"2024-09-21T01:21:40.830302+0000","last_fullsized":"2024-09-21T01:21:40.830302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T01:19:05.684126+0000","last_clean_scrub_stamp":"2024-09-21T01:19:05.684126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:25.373499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":870306,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1700,"num_read_kb":1093,"num_write":497,"num_write_kb":2259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":67516,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2985984,"data_stored":2610918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":377,"ondisk_log_size":377,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678071,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":535509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874299,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29192,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715960,"statfs":{"total":95995035648,"available":95965143040,"internally_reserved":0,"allocated":2158592,"data_stored":1595721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037825,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1466368,"data_stored":928521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234054,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1449984,"data_stored":928765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430284,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1133555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593811,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":531200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790041,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29108,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716044,"statfs":{"total":95995035648,"available":95965229056,"internally_reserved":0,"allocated":2072576,"data_stored":1521990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":609191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37212,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707940,"statfs":{"total":95995035648,"available":95956930560,"internally_reserved":0,"allocated":1851392,"data_stored":1317509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378729,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":531732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542254,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":607356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705780,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":532265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":77269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":77528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":471170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T01:22:26.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:22:26.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:26 smithi027 ceph-mon[30375]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:26.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:26 smithi089 ceph-mon[34486]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:26 smithi098 ceph-mon[34372]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T01:22:27.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.654+0000 7f746a800640 1 -- 172.21.15.27:0/447125214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464103990 msgr2=0x7f7464103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.654+0000 7f746a800640 1 --2- 172.21.15.27:0/447125214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464103990 0x7f7464103da0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7458009900 tx=0x7f745802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 -- 172.21.15.27:0/447125214 shutdown_connections 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 --2- 172.21.15.27:0/447125214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464109c20 0x7f7464110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 --2- 172.21.15.27:0/447125214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f74641096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 --2- 172.21.15.27:0/447125214 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464103990 0x7f7464103da0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 -- 172.21.15.27:0/447125214 >> 172.21.15.27:0/447125214 conn(0x7f74640ff3e0 msgr2=0x7f7464101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 -- 172.21.15.27:0/447125214 shutdown_connections 2024-09-21T01:22:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.655+0000 7f746a800640 1 -- 172.21.15.27:0/447125214 wait complete. 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 Processor -- start 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 -- start start 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464103990 0x7f746419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f746419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464109c20 0x7f746419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f74641138e0 con 0x7f74641092d0 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7464113760 con 0x7f7464109c20 2024-09-21T01:22:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.656+0000 7f746a800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7464113a60 con 0x7f7464103990 2024-09-21T01:22:27.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7468a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f746419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464103990 0x7f746419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464109c20 0x7f746419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7468a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f746419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58650/0 (socket says 172.21.15.27:58650) 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7468a00640 1 -- 172.21.15.27:0/3453081930 learned_addr learned my addr 172.21.15.27:0/3453081930 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469400640 1 -- 172.21.15.27:0/3453081930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464109c20 msgr2=0x7f746419fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469400640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464109c20 0x7f746419fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469400640 1 -- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 msgr2=0x7f746419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469400640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f746419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7469400640 1 -- 172.21.15.27:0/3453081930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74641a0200 con 0x7f7464103990 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.657+0000 7f7468a00640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f746419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.658+0000 7f7469400640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464103990 0x7f746419a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7458006010 tx=0x7f7458004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.658+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745803f070 con 0x7f7464103990 2024-09-21T01:22:27.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.658+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74641a0490 con 0x7f7464103990 2024-09-21T01:22:27.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.658+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7458003ca0 con 0x7f7464103990 2024-09-21T01:22:27.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.658+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458035070 con 0x7f7464103990 2024-09-21T01:22:27.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.658+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74641a6c50 con 0x7f7464103990 2024-09-21T01:22:27.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.659+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74440052c0 con 0x7f7464103990 2024-09-21T01:22:27.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.662+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f74580037f0 con 0x7f7464103990 2024-09-21T01:22:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.663+0000 7f7462000640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f744c0b6680 0x7f744c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.664+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7458107fb0 con 0x7f7464103990 2024-09-21T01:22:27.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.664+0000 7f7468a00640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f744c0b6680 0x7f744c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:27.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.664+0000 7f7468a00640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f744c0b6680 0x7f744c0b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7464108e30 tx=0x7f7450005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:27.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.665+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74580d2510 con 0x7f7464103990 2024-09-21T01:22:27.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.905+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74440055b0 con 0x7f7464103990 2024-09-21T01:22:27.906 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:27 smithi027 ceph-mon[30375]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-21T01:22:27.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.906+0000 7f7462000640 1 -- 172.21.15.27:0/3453081930 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f74580d7450 con 0x7f7464103990 2024-09-21T01:22:27.907 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:27.907 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:22:27.907 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f744c0b6680 msgr2=0x7f744c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f744c0b6680 0x7f744c0b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7464108e30 tx=0x7f7450005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464103990 msgr2=0x7f746419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464103990 0x7f746419a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7458006010 tx=0x7f7458004060 comp rx=0 tx=0).stop 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 shutdown_connections 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f744c0b6680 0x7f744c0b8b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7464109c20 0x7f746419fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74641092d0 0x7f746419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 --2- 172.21.15.27:0/3453081930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7464103990 0x7f746419a290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:27.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.908+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 >> 172.21.15.27:0/3453081930 conn(0x7f74640ff3e0 msgr2=0x7f7464101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:27.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.909+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 shutdown_connections 2024-09-21T01:22:27.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:27.909+0000 7f746a800640 1 -- 172.21.15.27:0/3453081930 wait complete. 2024-09-21T01:22:28.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:27 smithi089 ceph-mon[34486]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-21T01:22:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:27 smithi098 ceph-mon[34372]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-21T01:22:28.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:22:28.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:28 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3453081930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:29.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:28 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3453081930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:29.095 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:29.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:28 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3453081930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.773+0000 7f7621800640 1 -- 172.21.15.27:0/3470366986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 msgr2=0x7f761c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.773+0000 7f7621800640 1 --2- 172.21.15.27:0/3470366986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c10bad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7604009860 tx=0x7f760402a340 comp rx=0 tx=0).stop 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 -- 172.21.15.27:0/3470366986 shutdown_connections 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 --2- 172.21.15.27:0/3470366986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f761c10c010 0x7f761c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 --2- 172.21.15.27:0/3470366986 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c10bad0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 --2- 172.21.15.27:0/3470366986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 -- 172.21.15.27:0/3470366986 >> 172.21.15.27:0/3470366986 conn(0x7f761c101810 msgr2=0x7f761c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 -- 172.21.15.27:0/3470366986 shutdown_connections 2024-09-21T01:22:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.774+0000 7f7621800640 1 -- 172.21.15.27:0/3470366986 wait complete. 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 Processor -- start 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 -- start start 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f761c10c010 0x7f761c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f761c115cd0 con 0x7f761c105d80 2024-09-21T01:22:29.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f761c115b50 con 0x7f761c10b6c0 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.775+0000 7f7621800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f761c115e50 con 0x7f761c10c010 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f761aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f761aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58676/0 (socket says 172.21.15.27:58676) 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49092/0 (socket says 172.21.15.27:49092) 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f761aa00640 1 -- 172.21.15.27:0/601033044 learned_addr learned my addr 172.21.15.27:0/601033044 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:29.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f761b400640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f761c10c010 0x7f761c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 -- 172.21.15.27:0/601033044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f761c10c010 msgr2=0x7f761c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f761c10c010 0x7f761c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 -- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 msgr2=0x7f761c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 -- 172.21.15.27:0/601033044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c1a2630 con 0x7f761c10b6c0 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f761aa00640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.776+0000 7f7612000640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c19cc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7604009780 tx=0x7f7604032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.777+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604037040 con 0x7f761c10b6c0 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.777+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7604030c10 con 0x7f761c10b6c0 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.777+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c1a28c0 con 0x7f761c10b6c0 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.777+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76040035f0 con 0x7f761c10b6c0 2024-09-21T01:22:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.777+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a2d10 con 0x7f761c10b6c0 2024-09-21T01:22:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.778+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c106190 con 0x7f761c10b6c0 2024-09-21T01:22:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.781+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f7604043020 con 0x7f761c10b6c0 2024-09-21T01:22:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.782+0000 7f7613e00640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f76000b6680 0x7f76000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.782+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7604106eb0 con 0x7f761c10b6c0 2024-09-21T01:22:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.782+0000 7f761aa00640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f76000b6680 0x7f76000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.783+0000 7f761aa00640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f76000b6680 0x7f76000b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f760c0096c0 tx=0x7f760c009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:29.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:29.784+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f760404a060 con 0x7f761c10b6c0 2024-09-21T01:22:29.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:29 smithi027 ceph-mon[30375]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T01:22:30.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:29 smithi089 ceph-mon[34486]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T01:22:30.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.021+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f761c02e640 con 0x7f761c10b6c0 2024-09-21T01:22:30.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.022+0000 7f7613e00640 1 -- 172.21.15.27:0/601033044 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f76040cd0b0 con 0x7f761c10b6c0 2024-09-21T01:22:30.023 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:30.023 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:22:30.023 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f76000b6680 msgr2=0x7f76000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f76000b6680 0x7f76000b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f760c0096c0 tx=0x7f760c009260 comp rx=0 tx=0).stop 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 msgr2=0x7f761c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c19cc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7604009780 tx=0x7f7604032d50 comp rx=0 tx=0).stop 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 shutdown_connections 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f76000b6680 0x7f76000b8b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f761c10c010 0x7f761c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f761c10b6c0 0x7f761c19cc60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 --2- 172.21.15.27:0/601033044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f761c105d80 0x7f761c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 >> 172.21.15.27:0/601033044 conn(0x7f761c101810 msgr2=0x7f761c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 shutdown_connections 2024-09-21T01:22:30.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:30.025+0000 7f7621800640 1 -- 172.21.15.27:0/601033044 wait complete. 2024-09-21T01:22:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:29 smithi098 ceph-mon[34372]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T01:22:30.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:22:31.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:30 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/601033044' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:31.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:30 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/601033044' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:31.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:30 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/601033044' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:31.254 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 -- 172.21.15.27:0/3264551923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7494105120 msgr2=0x7f749410d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 --2- 172.21.15.27:0/3264551923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7494105120 0x7f749410d670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7480009900 tx=0x7f748002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:22:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 -- 172.21.15.27:0/3264551923 shutdown_connections 2024-09-21T01:22:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 --2- 172.21.15.27:0/3264551923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f749410dc30 0x7f7494110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:31.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 --2- 172.21.15.27:0/3264551923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7494105120 0x7f749410d670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 --2- 172.21.15.27:0/3264551923 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74941047d0 0x7f7494104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 -- 172.21.15.27:0/3264551923 >> 172.21.15.27:0/3264551923 conn(0x7f74940fe2d0 msgr2=0x7f7494100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 -- 172.21.15.27:0/3264551923 shutdown_connections 2024-09-21T01:22:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.899+0000 7f749a200640 1 -- 172.21.15.27:0/3264551923 wait complete. 2024-09-21T01:22:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.901+0000 7f749a200640 1 Processor -- start 2024-09-21T01:22:31.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.901+0000 7f749a200640 1 -- start start 2024-09-21T01:22:31.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.901+0000 7f749a200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74941047d0 0x7f749419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.901+0000 7f749a200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7494105120 0x7f749419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.901+0000 7f749a200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 0x7f749419fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f749a200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f74941138e0 con 0x7f749410dc30 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f749a200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7494113760 con 0x7f74941047d0 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f749a200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7494113a60 con 0x7f7494105120 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 0x7f749419fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 0x7f749419fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58688/0 (socket says 172.21.15.27:58688) 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 -- 172.21.15.27:0/588895046 learned_addr learned my addr 172.21.15.27:0/588895046 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7498e00640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74941047d0 0x7f749419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:31.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7493e00640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7494105120 0x7f749419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 -- 172.21.15.27:0/588895046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7494105120 msgr2=0x7f749419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7494105120 0x7f749419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 -- 172.21.15.27:0/588895046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74941047d0 msgr2=0x7f749419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74941047d0 0x7f749419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:31.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.902+0000 7f7499800640 1 -- 172.21.15.27:0/588895046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74941a0200 con 0x7f749410dc30 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.903+0000 7f7499800640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 0x7f749419fad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f748c00eb10 tx=0x7f748c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.903+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c004b20 con 0x7f749410dc30 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.903+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c004cc0 con 0x7f749410dc30 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.903+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74941a04f0 con 0x7f749410dc30 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.903+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c010670 con 0x7f749410dc30 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.903+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74941a6c50 con 0x7f749410dc30 2024-09-21T01:22:31.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.904+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7474005290 con 0x7f749410dc30 2024-09-21T01:22:31.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.907+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f748c005020 con 0x7f749410dc30 2024-09-21T01:22:31.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.908+0000 7f7491600640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f747c0b6700 0x7f747c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:31.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.908+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f748c0e7dd0 con 0x7f749410dc30 2024-09-21T01:22:31.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.909+0000 7f7498e00640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f747c0b6700 0x7f747c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:31.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.909+0000 7f7498e00640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f747c0b6700 0x7f747c0b8c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7488009bd0 tx=0x7f7488005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:31.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:31.912+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f748c0adf50 con 0x7f749410dc30 2024-09-21T01:22:32.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:31 smithi089 ceph-mon[34486]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-21T01:22:32.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.113+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7474005850 con 0x7f749410dc30 2024-09-21T01:22:32.113 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:31 smithi027 ceph-mon[30375]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-21T01:22:32.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.114+0000 7f7491600640 1 -- 172.21.15.27:0/588895046 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25150 (secure 0 0 0) 0x7f748c0b2330 con 0x7f749410dc30 2024-09-21T01:22:32.116 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:32.116 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":90,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:20:42.161651+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:22:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f747c0b6700 msgr2=0x7f747c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f747c0b6700 0x7f747c0b8c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7488009bd0 tx=0x7f7488005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 msgr2=0x7f749419fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 0x7f749419fad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f748c00eb10 tx=0x7f748c00c2d0 comp rx=0 tx=0).stop 2024-09-21T01:22:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 shutdown_connections 2024-09-21T01:22:32.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f747c0b6700 0x7f747c0b8c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f749410dc30 0x7f749419fad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7494105120 0x7f749419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 --2- 172.21.15.27:0/588895046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f74941047d0 0x7f749419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 >> 172.21.15.27:0/588895046 conn(0x7f74940fe2d0 msgr2=0x7f7494107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.117+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 shutdown_connections 2024-09-21T01:22:32.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:32.118+0000 7f749a200640 1 -- 172.21.15.27:0/588895046 wait complete. 2024-09-21T01:22:32.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:31 smithi098 ceph-mon[34372]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-21T01:22:32.831 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:32 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/588895046' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:22:32.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T01:22:33.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:32 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/588895046' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:22:33.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:32 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/588895046' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:22:33.307 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:34.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:33 smithi089 ceph-mon[34486]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 73 KiB/s wr, 18 op/s 2024-09-21T01:22:34.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:33 smithi027 ceph-mon[30375]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 73 KiB/s wr, 18 op/s 2024-09-21T01:22:34.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:33 smithi098 ceph-mon[34372]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 73 KiB/s wr, 18 op/s 2024-09-21T01:22:34.895 INFO:teuthology.orchestra.run.smithi027.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T01:22:34.895 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T01:22:34.896 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T01:22:34.896 DEBUG:tasks.quiescer:Yielding 2024-09-21T01:22:34.896 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T01:22:34.916 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:22:34.916 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T01:22:34.916 INFO:tasks.workunit:timeout=3h 2024-09-21T01:22:34.916 INFO:tasks.workunit:cleanup=True 2024-09-21T01:22:34.916 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:34.917 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-09-21 01:20:17.030556982 +0000 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-09-21 01:20:17.030556982 +0000 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-09-21 01:22:14.792603047 +0000 2024-09-21T01:22:34.955 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-09-21 01:20:17.030556982 +0000 2024-09-21T01:22:34.956 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T01:22:34.956 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T01:22:35.066 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T01:22:35.105 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T01:22:35.321 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:35.986 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:35 smithi027 ceph-mon[30375]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T01:22:35.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:35 smithi089 ceph-mon[34486]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T01:22:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 -- 172.21.15.27:0/1366375574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 msgr2=0x7fe6b410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/1366375574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b410bae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009900 tx=0x7fe6a002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 -- 172.21.15.27:0/1366375574 shutdown_connections 2024-09-21T01:22:36.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/1366375574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b410c020 0x7fe6b4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/1366375574 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b410bae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/1366375574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b4105d90 0x7fe6b41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 -- 172.21.15.27:0/1366375574 >> 172.21.15.27:0/1366375574 conn(0x7fe6b4101860 msgr2=0x7fe6b4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 -- 172.21.15.27:0/1366375574 shutdown_connections 2024-09-21T01:22:36.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.060+0000 7fe6b9c00640 1 -- 172.21.15.27:0/1366375574 wait complete. 2024-09-21T01:22:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.061+0000 7fe6b9c00640 1 Processor -- start 2024-09-21T01:22:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 -- start start 2024-09-21T01:22:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b4105d90 0x7fe6b419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b410c020 0x7fe6b41a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:36.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4115ce0 con 0x7fe6b410c020 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4115b60 con 0x7fe6b410b6d0 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b9c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b4115e60 con 0x7fe6b4105d90 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b4105d90 0x7fe6b419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b2000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42074/0 (socket says 172.21.15.27:42074) 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b2000640 1 -- 172.21.15.27:0/2739416566 learned_addr learned my addr 172.21.15.27:0/2739416566 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:36.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.062+0000 7fe6b3400640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b410c020 0x7fe6b41a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:36.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2000640 1 -- 172.21.15.27:0/2739416566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b4105d90 msgr2=0x7fe6b419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:36.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2000640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b4105d90 0x7fe6b419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2000640 1 -- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b410c020 msgr2=0x7fe6b41a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2000640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b410c020 0x7fe6b41a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2000640 1 -- 172.21.15.27:0/2739416566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b41a2600 con 0x7fe6b410b6d0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2a00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b4105d90 0x7fe6b419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b3400640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b410c020 0x7fe6b41a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6b2000640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b419cc30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe6a002a910 tx=0x7fe6a0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.063+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a003f030 con 0x7fe6b410b6d0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.064+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6a0004490 con 0x7fe6b410b6d0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.064+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b41a2890 con 0x7fe6b410b6d0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.064+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0035070 con 0x7fe6b410b6d0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.064+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b41a9050 con 0x7fe6b410b6d0 2024-09-21T01:22:36.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.065+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b41061a0 con 0x7fe6b410b6d0 2024-09-21T01:22:36.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.067+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7fe6a0042030 con 0x7fe6b410b6d0 2024-09-21T01:22:36.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.069+0000 7fe6ab400640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe6980b6700 0x7fe6980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:36.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.069+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe6a0108130 con 0x7fe6b410b6d0 2024-09-21T01:22:36.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.069+0000 7fe6b2a00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe6980b6700 0x7fe6980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:36.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.070+0000 7fe6b2a00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe6980b6700 0x7fe6980b8c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009fc0 tx=0x7fe6a4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:36.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.071+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6a00d2690 con 0x7fe6b410b6d0 2024-09-21T01:22:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:35 smithi098 ceph-mon[34372]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T01:22:36.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.310+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6b419d7c0 con 0x7fe6b410b6d0 2024-09-21T01:22:36.312 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:22:36.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.311+0000 7fe6ab400640 1 -- 172.21.15.27:0/2739416566 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fe6b419d7c0 con 0x7fe6b410b6d0 2024-09-21T01:22:36.313 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:22:36.313 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe6980b6700 msgr2=0x7fe6980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe6980b6700 0x7fe6980b8c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009fc0 tx=0x7fe6a4005f90 comp rx=0 tx=0).stop 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 msgr2=0x7fe6b419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b419cc30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe6a002a910 tx=0x7fe6a0031a20 comp rx=0 tx=0).stop 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 shutdown_connections 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe6980b6700 0x7fe6980b8c10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b410c020 0x7fe6b41a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe6b410b6d0 0x7fe6b419cc30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 --2- 172.21.15.27:0/2739416566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6b4105d90 0x7fe6b419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:36.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.314+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 >> 172.21.15.27:0/2739416566 conn(0x7fe6b4101860 msgr2=0x7fe6b4103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:36.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.315+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 shutdown_connections 2024-09-21T01:22:36.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:36.315+0000 7fe6b9c00640 1 -- 172.21.15.27:0/2739416566 wait complete. 2024-09-21T01:22:37.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:36 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2739416566' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:37.091 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:36 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2739416566' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:37.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T01:19:07.935504+0000', 'modified': '2024-09-21T01:22:15.795685+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24776, 'mds_1': 24752, 'mds_2': 24734, 'mds_3': 24730, 'mds_4': 24718}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6833/39744273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6832', 'nonce': 39744273}, {'type': 'v1', 'addr': '172.21.15.89:6833', 'nonce': 39744273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24686': {'gid': 24686, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6835/2864798916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 2864798916}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2864798916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24695': {'gid': 24695, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6837/392681445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6836', 'nonce': 392681445}, {'type': 'v1', 'addr': '172.21.15.27:6837', 'nonce': 392681445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24706': {'gid': 24706, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/1764678372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 1764678372}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 1764678372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6835/3850699909', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6834', 'nonce': 3850699909}, {'type': 'v1', 'addr': '172.21.15.89:6835', 'nonce': 3850699909}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24718': {'gid': 24718, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.98:6837/1081631404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1081631404}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1081631404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24730': {'gid': 24730, 'name': 'i', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.98:6839/3799755243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 3799755243}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 3799755243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24734': {'gid': 24734, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.89:6837/3556269004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6836', 'nonce': 3556269004}, {'type': 'v1', 'addr': '172.21.15.89:6837', 'nonce': 3556269004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24752': {'gid': 24752, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.27:6839/3115497172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3115497172}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3115497172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24776': {'gid': 24776, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.89:6839/4156411896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 4156411896}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 4156411896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24776, 'qdb_cluster': [24776, 24752, 24734, 24730, 24718]} 2024-09-21T01:22:37.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T01:22:37.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T01:22:37.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:36 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2739416566' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:22:37.558 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:38.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:37 smithi089 ceph-mon[34486]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:22:38.012 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:37 smithi027 ceph-mon[30375]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:22:38.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:37 smithi098 ceph-mon[34372]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 -- 172.21.15.27:0/3081040683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 msgr2=0x7f9d841098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 --2- 172.21.15.27:0/3081040683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 0x7f9d841098b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009860 tx=0x7f9d6c02a340 comp rx=0 tx=0).stop 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 -- 172.21.15.27:0/3081040683 shutdown_connections 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 --2- 172.21.15.27:0/3081040683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84109df0 0x7f9d84110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 --2- 172.21.15.27:0/3081040683 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 0x7f9d841098b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 --2- 172.21.15.27:0/3081040683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84103b60 0x7f9d84103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 -- 172.21.15.27:0/3081040683 >> 172.21.15.27:0/3081040683 conn(0x7f9d8407b850 msgr2=0x7f9d8407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.196+0000 7f9d89600640 1 -- 172.21.15.27:0/3081040683 shutdown_connections 2024-09-21T01:22:38.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.197+0000 7f9d89600640 1 -- 172.21.15.27:0/3081040683 wait complete. 2024-09-21T01:22:38.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.198+0000 7f9d89600640 1 Processor -- start 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.198+0000 7f9d89600640 1 -- start start 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d89600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84103b60 0x7f9d8419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d89600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 0x7f9d8419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d89600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 0x7f9d8419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d89600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84113ab0 con 0x7f9d84109df0 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d89600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84113930 con 0x7f9d841094a0 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d89600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d84113c30 con 0x7f9d84103b60 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 0x7f9d8419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 0x7f9d8419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55490/0 (socket says 172.21.15.27:55490) 2024-09-21T01:22:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d83400640 1 -- 172.21.15.27:0/3522174703 learned_addr learned my addr 172.21.15.27:0/3522174703 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d82a00640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84103b60 0x7f9d8419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.199+0000 7f9d7a000640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 0x7f9d8419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d83400640 1 -- 172.21.15.27:0/3522174703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84103b60 msgr2=0x7f9d8419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d83400640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84103b60 0x7f9d8419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d83400640 1 -- 172.21.15.27:0/3522174703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 msgr2=0x7f9d8419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d83400640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 0x7f9d8419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d83400640 1 -- 172.21.15.27:0/3522174703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d841a0450 con 0x7f9d84109df0 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d82a00640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84103b60 0x7f9d8419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d83400640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 0x7f9d8419fd20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00eb10 tx=0x7f9d7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d7be00640 1 -- 172.21.15.27:0/3522174703 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c004b20 con 0x7f9d84109df0 2024-09-21T01:22:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d7be00640 1 -- 172.21.15.27:0/3522174703 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c004cc0 con 0x7f9d84109df0 2024-09-21T01:22:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d841a0740 con 0x7f9d84109df0 2024-09-21T01:22:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.200+0000 7f9d7be00640 1 -- 172.21.15.27:0/3522174703 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c010670 con 0x7f9d84109df0 2024-09-21T01:22:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.201+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841a6ea0 con 0x7f9d84109df0 2024-09-21T01:22:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.204+0000 7f9d90255740 1 Processor -- start 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.204+0000 7f9d7be00640 1 -- 172.21.15.27:0/3522174703 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f9d7c005020 con 0x7f9d84109df0 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.204+0000 7f9d90255740 1 -- start start 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d90255740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e5262f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d90255740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5262fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d90255740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e52630320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d90255740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e525a2d50 con 0x561e52264280 2024-09-21T01:22:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d90255740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561e52592180 con 0x561e5225c0a0 2024-09-21T01:22:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d90255740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e52595f90 con 0x561e52284f60 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d7a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e52630320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d82a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5262fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d7a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e52630320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55502/0 (socket says 172.21.15.27:55502) 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d7a000640 1 -- 172.21.15.27:0/3901772567 learned_addr learned my addr 172.21.15.27:0/3901772567 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.205+0000 7f9d7a000640 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e5262f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d82a00640 1 -- 172.21.15.27:0/3901772567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 msgr2=0x561e5262f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d82a00640 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e5262f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d82a00640 1 -- 172.21.15.27:0/3901772567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 msgr2=0x561e52630320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d82a00640 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e52630320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d82a00640 1 -- 172.21.15.27:0/3901772567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5255ca70 con 0x561e5225c0a0 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d7be00640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9d680b6700 0x7f9d680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d82a00640 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5262fde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005fd0 tx=0x7f9d740118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdea761ee03bd88d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d7be00640 1 -- 172.21.15.27:0/3522174703 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9d7c0e7fe0 con 0x7f9d84109df0 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d7a000640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9d680b6700 0x7f9d680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.206+0000 7f9d73400640 1 -- 172.21.15.27:0/3901772567 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74004020 con 0x561e5225c0a0 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.207+0000 7f9d73400640 1 -- 172.21.15.27:0/3901772567 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d740041c0 con 0x561e5225c0a0 2024-09-21T01:22:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.207+0000 7f9d73400640 1 -- 172.21.15.27:0/3901772567 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400f810 con 0x561e5225c0a0 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.207+0000 7f9d90255740 1 -- 172.21.15.27:0/3901772567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 msgr2=0x561e5262fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.207+0000 7f9d90255740 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5262fde0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005fd0 tx=0x7f9d740118d0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.207+0000 7f9d7a000640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9d680b6700 0x7f9d680b8c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c02cfd0 tx=0x7f9d6c005780 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 -- 172.21.15.27:0/3901772567 shutdown_connections 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e52630320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5262fde0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 --2- 172.21.15.27:0/3901772567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e5262f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 -- 172.21.15.27:0/3901772567 >> 172.21.15.27:0/3901772567 conn(0x561e52279350 msgr2=0x561e5262d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 -- 172.21.15.27:0/3901772567 shutdown_connections 2024-09-21T01:22:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 -- 172.21.15.27:0/3901772567 wait complete. 2024-09-21T01:22:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.208+0000 7f9d90255740 1 -- start start 2024-09-21T01:22:38.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 Processor -- start 2024-09-21T01:22:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5263b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e5263b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e526473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e523f35f0 con 0x561e52264280 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561e5252d460 con 0x561e5225c0a0 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e5263e080 con 0x561e52284f60 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.209+0000 7f9d90255740 10 client.0 _ll_register_callbacks cb 0x561e5253d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d82a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5263b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d7a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e5263b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d83400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e526473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d82a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5263b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42084/0 (socket says 172.21.15.27:42084) 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d82a00640 1 -- 172.21.15.27:0/2284329149 learned_addr learned my addr 172.21.15.27:0/2284329149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d7a000640 1 -- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 msgr2=0x561e526473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d7a000640 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e526473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d7a000640 1 -- 172.21.15.27:0/2284329149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 msgr2=0x561e5263b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d7a000640 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5263b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.210+0000 7f9d7a000640 1 -- 172.21.15.27:0/2284329149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e522f0540 con 0x561e52264280 2024-09-21T01:22:38.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d7a000640 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e5263b920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0095f0 tx=0x7f9d6c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd172beb8434666 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:38.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d67e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:38.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d67e00640 1 -- 172.21.15.27:0/2284329149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c04e070 con 0x561e52264280 2024-09-21T01:22:38.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e5252fa70 con 0x561e52264280 2024-09-21T01:22:38.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d67e00640 1 -- 172.21.15.27:0/2284329149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9d6c030860 con 0x561e52264280 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d67e00640 1 -- 172.21.15.27:0/2284329149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d6c05a050 con 0x561e52264280 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d90255740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.211+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e5254a6f0 con 0x561e52264280 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d67e00640 1 -- 172.21.15.27:0/2284329149 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561e5252fa70 con 0x561e52264280 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d67e00640 1 -- 172.21.15.27:0/2284329149 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9d6c049040 con 0x561e52264280 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d90255740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d90255740 10 client.15513 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:22:38.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d90255740 20 client.15513 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d90255740 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e522438e0 0x561e5264d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d90255740 4 client.15513 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.212+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 --> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e525a4ca0 con 0x561e522438e0 2024-09-21T01:22:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.213+0000 7f9d82a00640 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e522438e0 0x561e5264d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.213+0000 7f9d82a00640 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e522438e0 0x561e5264d8c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=823b4455791bd258 server_cookie=820a1a584268f65e in_seq=0 out_seq=0 2024-09-21T01:22:38.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.213+0000 7f9d67e00640 10 client.15513 ms_handle_connect on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.214+0000 7f9d67e00640 1 -- 172.21.15.27:0/2284329149 <== mds.4 v2:172.21.15.98:6836/1081631404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e525a4ca0 con 0x561e522438e0 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.214+0000 7f9d67e00640 10 client.15513 handle_command_reply: tid=0 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.214+0000 7f9d90255740 1 client.15513 shutdown 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 msgr2=0x561e5263b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e5263b920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c0095f0 tx=0x7f9d6c02a340 comp rx=0 tx=0).stop 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 shutdown_connections 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e522438e0 0x561e5264d8c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52284f60 0x561e526473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52264280 0x561e5263b920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 --2- 172.21.15.27:0/2284329149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5263b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d67e00640 0 client.15513 ms_handle_reset on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 >> 172.21.15.27:0/2284329149 conn(0x561e52279350 msgr2=0x561e52632a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 shutdown_connections 2024-09-21T01:22:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 1 -- 172.21.15.27:0/2284329149 wait complete. 2024-09-21T01:22:38.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.215+0000 7f9d90255740 20 client.15513 trim_cache size 0 max 16384 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 Processor -- start 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 -- start start 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5264d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52264280 0x561e52624010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e52636390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e5264bc00 con 0x561e52284f60 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561e5264c2d0 con 0x561e5225c0a0 2024-09-21T01:22:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.234+0000 7f9d90255740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e5264c9a0 con 0x561e52264280 2024-09-21T01:22:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e52636390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d7a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52264280 0x561e52624010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e52636390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55518/0 (socket says 172.21.15.27:55518) 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 -- 172.21.15.27:0/682883745 learned_addr learned my addr 172.21.15.27:0/682883745 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d82a00640 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5264d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 -- 172.21.15.27:0/682883745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52264280 msgr2=0x561e52624010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52264280 0x561e52624010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 -- 172.21.15.27:0/682883745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 msgr2=0x561e5264d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5264d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d83400640 1 -- 172.21.15.27:0/682883745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5263dbb0 con 0x561e52284f60 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.235+0000 7f9d82a00640 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5264d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.236+0000 7f9d83400640 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e52636390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005ba0 tx=0x7f9d7c0a99c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d7517136edb4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.236+0000 7f9d72a00640 1 -- 172.21.15.27:0/682883745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0a91d0 con 0x561e52284f60 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.236+0000 7f9d72a00640 1 -- 172.21.15.27:0/682883745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c0ad570 con 0x561e52284f60 2024-09-21T01:22:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.236+0000 7f9d72a00640 1 -- 172.21.15.27:0/682883745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b20d0 con 0x561e52284f60 2024-09-21T01:22:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 -- 172.21.15.27:0/682883745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 msgr2=0x561e52636390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e52636390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005ba0 tx=0x7f9d7c0a99c0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 -- 172.21.15.27:0/682883745 shutdown_connections 2024-09-21T01:22:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e52636390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e52264280 0x561e52624010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 --2- 172.21.15.27:0/682883745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e5225c0a0 0x561e5264d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 -- 172.21.15.27:0/682883745 >> 172.21.15.27:0/682883745 conn(0x561e52279350 msgr2=0x561e52647520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 -- 172.21.15.27:0/682883745 shutdown_connections 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 -- 172.21.15.27:0/682883745 wait complete. 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.237+0000 7f9d90255740 1 -- start start 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 Processor -- start 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e5225c0a0 0x561e52656440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e52264280 0x561e526500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e526506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561e5264d740 con 0x561e52284f60 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561e5264d070 con 0x561e52264280 2024-09-21T01:22:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561e5264de10 con 0x561e5225c0a0 2024-09-21T01:22:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d90255740 10 client.0 _ll_register_callbacks cb 0x7f9d6c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e526506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d83400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e526506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55532/0 (socket says 172.21.15.27:55532) 2024-09-21T01:22:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d83400640 1 -- 172.21.15.27:0/777449239 learned_addr learned my addr 172.21.15.27:0/777449239 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d7a000640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e52264280 0x561e526500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.238+0000 7f9d82a00640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e5225c0a0 0x561e52656440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d83400640 1 -- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e5225c0a0 msgr2=0x561e52656440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d83400640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e5225c0a0 0x561e52656440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d83400640 1 -- 172.21.15.27:0/777449239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e52264280 msgr2=0x561e526500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d83400640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e52264280 0x561e526500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d83400640 1 -- 172.21.15.27:0/777449239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e52650e80 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d7a000640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e52264280 0x561e526500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d83400640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e526506a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0d5170 tx=0x7f9d7c0a9a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cb9b652762f699 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d67e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d67e00640 1 -- 172.21.15.27:0/777449239 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0c7020 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d67e00640 1 -- 172.21.15.27:0/777449239 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9d7c0b22d0 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.239+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c0ad570 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d67e00640 1 -- 172.21.15.27:0/777449239 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c0ae160 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561e526b8d90 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d67e00640 1 -- 172.21.15.27:0/777449239 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d7c0ad570 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d67e00640 1 -- 172.21.15.27:0/777449239 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9d7c0a93b0 con 0x561e52284f60 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 10 client.15519 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 20 client.15519 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e52218c90 0x561e526bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 4 client.15519 mds_command: new command op to 24718 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.240+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 --> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] -- command(tid 0: {"prefix": "status"}) -- 0x561e525a0750 con 0x561e52218c90 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.241+0000 7f9d82a00640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e52218c90 0x561e526bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.241+0000 7f9d82a00640 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e52218c90 0x561e526bc190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fdc4692d5c9c8617 server_cookie=df07f6018b341b3 in_seq=0 out_seq=0 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.241+0000 7f9d67e00640 10 client.15519 ms_handle_connect on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.241+0000 7f9d67e00640 1 -- 172.21.15.27:0/777449239 <== mds.4 v2:172.21.15.98:6836/1081631404 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561e525a0750 con 0x561e52218c90 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.241+0000 7f9d67e00640 10 client.15519 handle_command_reply: tid=0 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.241+0000 7f9d90255740 1 client.15519 shutdown 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 msgr2=0x561e526506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e526506a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0d5170 tx=0x7f9d7c0a9a30 comp rx=0 tx=0).stop 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 shutdown_connections 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:6836/1081631404,v1:172.21.15.98:6837/1081631404] conn(0x561e52218c90 0x561e526bc190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561e52284f60 0x561e526506a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561e52264280 0x561e526500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 --2- 172.21.15.27:0/777449239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561e5225c0a0 0x561e52656440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d67e00640 0 client.15519 ms_handle_reset on v2:172.21.15.98:6836/1081631404 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 >> 172.21.15.27:0/777449239 conn(0x561e52279350 msgr2=0x561e5263b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 shutdown_connections 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 1 -- 172.21.15.27:0/777449239 wait complete. 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.242+0000 7f9d90255740 20 client.15519 trim_cache size 0 max 16384 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24718, 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:38.246 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 153.86736323700001, 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 240.18647515699999 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:38.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9d680b6700 msgr2=0x7f9d680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9d680b6700 0x7f9d680b8c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c02cfd0 tx=0x7f9d6c005780 comp rx=0 tx=0).stop 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 msgr2=0x7f9d8419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 0x7f9d8419fd20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00eb10 tx=0x7f9d7c00c2d0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 shutdown_connections 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f9d680b6700 0x7f9d680b8c10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d84109df0 0x7f9d8419fd20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9d841094a0 0x7f9d8419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 --2- 172.21.15.27:0/3522174703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d84103b60 0x7f9d8419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 >> 172.21.15.27:0/3522174703 conn(0x7f9d8407b850 msgr2=0x7f9d841075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 shutdown_connections 2024-09-21T01:22:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:38.243+0000 7f9d89600640 1 -- 172.21.15.27:0/3522174703 wait complete. 2024-09-21T01:22:39.192 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T01:22:39.577 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:40.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:39 smithi089 ceph-mon[34486]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T01:22:40.040 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:39 smithi027 ceph-mon[30375]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T01:22:40.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:39 smithi098 ceph-mon[34372]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-21T01:22:40.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 -- 172.21.15.27:0/3220009739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe40757b0 msgr2=0x7fdbe4075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 --2- 172.21.15.27:0/3220009739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe40757b0 0x7fdbe4075bc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009900 tx=0x7fdbd802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 -- 172.21.15.27:0/3220009739 shutdown_connections 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 --2- 172.21.15.27:0/3220009739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe410ff10 0x7fdbe4112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 --2- 172.21.15.27:0/3220009739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 0x7fdbe410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 --2- 172.21.15.27:0/3220009739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe40757b0 0x7fdbe4075bc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 -- 172.21.15.27:0/3220009739 >> 172.21.15.27:0/3220009739 conn(0x7fdbe40ff440 msgr2=0x7fdbe4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 -- 172.21.15.27:0/3220009739 shutdown_connections 2024-09-21T01:22:40.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.222+0000 7fdbea000640 1 -- 172.21.15.27:0/3220009739 wait complete. 2024-09-21T01:22:40.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.223+0000 7fdbea000640 1 Processor -- start 2024-09-21T01:22:40.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.223+0000 7fdbea000640 1 -- start start 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbea000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe40757b0 0x7fdbe419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbea000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 0x7fdbe419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbea000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe410ff10 0x7fdbe41a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbea000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4115ac0 con 0x7fdbe410ff10 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbea000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4115940 con 0x7fdbe4076100 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbea000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4115c40 con 0x7fdbe40757b0 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe40757b0 0x7fdbe419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe40757b0 0x7fdbe419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49828/0 (socket says 172.21.15.27:49828) 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe8c00640 1 -- 172.21.15.27:0/3810941482 learned_addr learned my addr 172.21.15.27:0/3810941482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe9600640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe410ff10 0x7fdbe41a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 0x7fdbe419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe9600640 1 -- 172.21.15.27:0/3810941482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe40757b0 msgr2=0x7fdbe419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.224+0000 7fdbe9600640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe40757b0 0x7fdbe419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe9600640 1 -- 172.21.15.27:0/3810941482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 msgr2=0x7fdbe419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe9600640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 0x7fdbe419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe9600640 1 -- 172.21.15.27:0/3810941482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe41a2420 con 0x7fdbe410ff10 2024-09-21T01:22:40.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 0x7fdbe419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:40.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe9600640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe410ff10 0x7fdbe41a1cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009fd0 tx=0x7fdbdc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:40.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe1600640 1 -- 172.21.15.27:0/3810941482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00ce20 con 0x7fdbe410ff10 2024-09-21T01:22:40.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe1600640 1 -- 172.21.15.27:0/3810941482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbdc005020 con 0x7fdbe410ff10 2024-09-21T01:22:40.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbe1600640 1 -- 172.21.15.27:0/3810941482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc010690 con 0x7fdbe410ff10 2024-09-21T01:22:40.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe41a2710 con 0x7fdbe410ff10 2024-09-21T01:22:40.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.225+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe41a8e70 con 0x7fdbe410ff10 2024-09-21T01:22:40.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.229+0000 7fdbf2238740 1 Processor -- start 2024-09-21T01:22:40.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.229+0000 7fdbe1600640 1 -- 172.21.15.27:0/3810941482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7fdbdc0054d0 con 0x7fdbe410ff10 2024-09-21T01:22:40.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.229+0000 7fdbf2238740 1 -- start start 2024-09-21T01:22:40.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbf2238740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55987b31ed50 con 0x55987b000f60 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbf2238740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55987b30e180 con 0x55987afd80a0 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbf2238740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55987b311f90 con 0x55987afe0280 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55560/0 (socket says 172.21.15.27:55560) 2024-09-21T01:22:40.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbe3e00640 1 -- 172.21.15.27:0/1639096898 learned_addr learned my addr 172.21.15.27:0/1639096898 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.230+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe8c00640 1 -- 172.21.15.27:0/1639096898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 msgr2=0x55987b3a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe8c00640 1 -- 172.21.15.27:0/1639096898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 msgr2=0x55987b3a7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe8c00640 1 -- 172.21.15.27:0/1639096898 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987b2d8a70 con 0x55987afd80a0 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3a7bb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8004770 tx=0x7fdbd8031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86890f9b06c4e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:40.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbd5c00640 1 -- 172.21.15.27:0/1639096898 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8040070 con 0x55987afd80a0 2024-09-21T01:22:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbd5c00640 1 -- 172.21.15.27:0/1639096898 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd8004030 con 0x55987afd80a0 2024-09-21T01:22:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.231+0000 7fdbd5c00640 1 -- 172.21.15.27:0/1639096898 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8035040 con 0x55987afd80a0 2024-09-21T01:22:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.232+0000 7fdbf2238740 1 -- 172.21.15.27:0/1639096898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 msgr2=0x55987b3a7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.232+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3a7bb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8004770 tx=0x7fdbd8031b10 comp rx=0 tx=0).stop 2024-09-21T01:22:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbe1600640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdbcc0b6700 0x7fdbcc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 -- 172.21.15.27:0/1639096898 shutdown_connections 2024-09-21T01:22:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3a7bb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1639096898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 -- 172.21.15.27:0/1639096898 >> 172.21.15.27:0/1639096898 conn(0x55987aff5350 msgr2=0x55987b3a5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:40.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbe1600640 1 -- 172.21.15.27:0/3810941482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdbdc0e8140 con 0x7fdbe410ff10 2024-09-21T01:22:40.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdbcc0b6700 0x7fdbcc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 -- 172.21.15.27:0/1639096898 shutdown_connections 2024-09-21T01:22:40.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.233+0000 7fdbf2238740 1 -- 172.21.15.27:0/1639096898 wait complete. 2024-09-21T01:22:40.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 -- start start 2024-09-21T01:22:40.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdbcc0b6700 0x7fdbcc0b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009580 tx=0x7fdbd0009c00 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:40.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 Processor -- start 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 0x55987b3b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afe0280 0x55987b3b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987b000f60 0x55987b3bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55987b16f5f0 con 0x55987afe0280 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55987b2a9460 con 0x55987b000f60 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.234+0000 7fdbf2238740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55987b3b5df0 con 0x55987afd80a0 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 0x55987b3b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbf2238740 10 client.0 _ll_register_callbacks cb 0x55987b2b9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:40.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 0x55987b3b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49850/0 (socket says 172.21.15.27:49850) 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 -- 172.21.15.27:0/584850059 learned_addr learned my addr 172.21.15.27:0/584850059 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afe0280 0x55987b3b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe9600640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987b000f60 0x55987b3bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 -- 172.21.15.27:0/584850059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987b000f60 msgr2=0x55987b3bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987b000f60 0x55987b3bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 -- 172.21.15.27:0/584850059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afe0280 msgr2=0x55987b3b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afe0280 0x55987b3b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.235+0000 7fdbe8c00640 1 -- 172.21.15.27:0/584850059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987b06c540 con 0x55987afd80a0 2024-09-21T01:22:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 0x55987b3b31c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8031440 tx=0x7fdbd802ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2efdea23846eb55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbca000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:22:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbca000640 1 -- 172.21.15.27:0/584850059 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd804b070 con 0x55987afd80a0 2024-09-21T01:22:40.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55987b2aba70 con 0x55987afd80a0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbca000640 1 -- 172.21.15.27:0/584850059 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdbd80314a0 con 0x55987afd80a0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbca000640 1 -- 172.21.15.27:0/584850059 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdbd8057050 con 0x55987afd80a0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbf2238740 10 client.25558 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.236+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55987b2c66f0 con 0x55987afd80a0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbca000640 1 -- 172.21.15.27:0/584850059 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55987b2aba70 con 0x55987afd80a0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbca000640 1 -- 172.21.15.27:0/584850059 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdbd8004a20 con 0x55987afd80a0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbf2238740 10 client.25558 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbf2238740 10 client.25558 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbf2238740 20 client.25558 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbf2238740 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987afbf8e0 0x55987b3c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbf2238740 4 client.25558 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.237+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 --> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55987b320ca0 con 0x55987afbf8e0 2024-09-21T01:22:40.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.238+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987afbf8e0 0x55987b3c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.238+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987afbf8e0 0x55987b3c5150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d5d0a21e9147e420 server_cookie=42389c5a9c60cc2f in_seq=0 out_seq=0 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.238+0000 7fdbca000640 10 client.25558 ms_handle_connect on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbca000640 1 -- 172.21.15.27:0/584850059 <== mds.3 v2:172.21.15.98:6838/3799755243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55987b320ca0 con 0x55987afbf8e0 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbca000640 10 client.25558 handle_command_reply: tid=0 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 client.25558 shutdown 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 msgr2=0x55987b3b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 0x55987b3b31c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8031440 tx=0x7fdbd802ab70 comp rx=0 tx=0).stop 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 shutdown_connections 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987afbf8e0 0x55987b3c5150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987b000f60 0x55987b3bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afe0280 0x55987b3b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 --2- 172.21.15.27:0/584850059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afd80a0 0x55987b3b31c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbca000640 0 client.25558 ms_handle_reset on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:40.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.239+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 >> 172.21.15.27:0/584850059 conn(0x55987aff5350 msgr2=0x55987b3aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:40.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.240+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 shutdown_connections 2024-09-21T01:22:40.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.240+0000 7fdbf2238740 1 -- 172.21.15.27:0/584850059 wait complete. 2024-09-21T01:22:40.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.240+0000 7fdbf2238740 20 client.25558 trim_cache size 0 max 16384 2024-09-21T01:22:40.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 Processor -- start 2024-09-21T01:22:40.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 -- start start 2024-09-21T01:22:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55987b3c3490 con 0x55987b000f60 2024-09-21T01:22:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55987b3c3b60 con 0x55987afd80a0 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.259+0000 7fdbf2238740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55987b3c4230 con 0x55987afe0280 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe9600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55566/0 (socket says 172.21.15.27:55566) 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49862/0 (socket says 172.21.15.27:49862) 2024-09-21T01:22:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe9600640 1 -- 172.21.15.27:0/3769410515 learned_addr learned my addr 172.21.15.27:0/3769410515 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 -- 172.21.15.27:0/3769410515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 msgr2=0x55987b3b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 -- 172.21.15.27:0/3769410515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 msgr2=0x55987b3a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 -- 172.21.15.27:0/3769410515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987b3b4060 con 0x55987afe0280 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe9600640 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.260+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3ad790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55987b3c5f90 tx=0x7fdbd0017040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b6fd6c8c9628e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.261+0000 7fdbd5200640 1 -- 172.21.15.27:0/3769410515 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0030070 con 0x55987afe0280 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.261+0000 7fdbd5200640 1 -- 172.21.15.27:0/3769410515 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd000fc00 con 0x55987afe0280 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.261+0000 7fdbd5200640 1 -- 172.21.15.27:0/3769410515 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0026070 con 0x55987afe0280 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.261+0000 7fdbf2238740 1 -- 172.21.15.27:0/3769410515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 msgr2=0x55987b3ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.261+0000 7fdbf2238740 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3ad790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55987b3c5f90 tx=0x7fdbd0017040 comp rx=0 tx=0).stop 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 -- 172.21.15.27:0/3769410515 shutdown_connections 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987b000f60 0x55987b3a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987afe0280 0x55987b3ad790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 --2- 172.21.15.27:0/3769410515 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afd80a0 0x55987b3b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 -- 172.21.15.27:0/3769410515 >> 172.21.15.27:0/3769410515 conn(0x55987aff5350 msgr2=0x55987b3a4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:40.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 -- 172.21.15.27:0/3769410515 shutdown_connections 2024-09-21T01:22:40.265 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 -- 172.21.15.27:0/3769410515 wait complete. 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 -- start start 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.262+0000 7fdbf2238740 1 Processor -- start 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afd80a0 0x55987b3cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 0x55987b3c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987b000f60 0x55987b3c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55987b3c4fd0 con 0x55987afd80a0 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55987b3c4900 con 0x55987afe0280 2024-09-21T01:22:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55987b3c56a0 con 0x55987b000f60 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbf2238740 10 client.0 _ll_register_callbacks cb 0x7fdbd8005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 0x55987b3c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbe9600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987b000f60 0x55987b3c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbe8c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afd80a0 0x55987b3cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbe3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 0x55987b3c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42166/0 (socket says 172.21.15.27:42166) 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.263+0000 7fdbe3e00640 1 -- 172.21.15.27:0/1740914389 learned_addr learned my addr 172.21.15.27:0/1740914389 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe9600640 1 -- 172.21.15.27:0/1740914389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 msgr2=0x55987b3c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe9600640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 0x55987b3c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe9600640 1 -- 172.21.15.27:0/1740914389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afd80a0 msgr2=0x55987b3cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe9600640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afd80a0 0x55987b3cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe9600640 1 -- 172.21.15.27:0/1740914389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987b3c7810 con 0x55987b000f60 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe3e00640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 0x55987b3c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afd80a0 0x55987b3cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbe9600640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987b000f60 0x55987b3c72d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0951b0 tx=0x7fdbdc0e5a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5170b620b7c8a107 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbca000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbca000640 1 -- 172.21.15.27:0/1740914389 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0d5030 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd000fc00 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbca000640 1 -- 172.21.15.27:0/1740914389 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdbdc0b7350 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbca000640 1 -- 172.21.15.27:0/1740914389 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc0ae2c0 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbf2238740 10 client.25564 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.264+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55987b3cd210 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbca000640 1 -- 172.21.15.27:0/1740914389 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd000fc00 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbca000640 1 -- 172.21.15.27:0/1740914389 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdbdc0b2260 con 0x55987b000f60 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbf2238740 10 client.25564 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbf2238740 10 client.25564 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbf2238740 20 client.25564 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987af94c90 0x55987b433b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbf2238740 4 client.25564 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.265+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 --> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] -- command(tid 0: {"prefix": "status"}) -- 0x55987b31c750 con 0x55987af94c90 2024-09-21T01:22:40.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.266+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987af94c90 0x55987b433b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:40.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.266+0000 7fdbe8c00640 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987af94c90 0x55987b433b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8efe473c73f6eb35 server_cookie=d333ac037df8eca4 in_seq=0 out_seq=0 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.266+0000 7fdbca000640 10 client.25564 ms_handle_connect on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.266+0000 7fdbca000640 1 -- 172.21.15.27:0/1740914389 <== mds.3 v2:172.21.15.98:6838/3799755243 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55987b31c750 con 0x55987af94c90 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.266+0000 7fdbca000640 10 client.25564 handle_command_reply: tid=0 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.266+0000 7fdbf2238740 1 client.25564 shutdown 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987b000f60 msgr2=0x55987b3c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987b000f60 0x55987b3c72d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0951b0 tx=0x7fdbdc0e5a40 comp rx=0 tx=0).stop 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 shutdown_connections 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:6838/3799755243,v1:172.21.15.98:6839/3799755243] conn(0x55987af94c90 0x55987b433b70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55987b000f60 0x55987b3c72d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55987afe0280 0x55987b3c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbca000640 0 client.25564 ms_handle_reset on v2:172.21.15.98:6838/3799755243 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 --2- 172.21.15.27:0/1740914389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55987afd80a0 0x55987b3cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 >> 172.21.15.27:0/1740914389 conn(0x55987aff5350 msgr2=0x55987b3a3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 shutdown_connections 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 1 -- 172.21.15.27:0/1740914389 wait complete. 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.267+0000 7fdbf2238740 20 client.25564 trim_cache size 0 max 16384 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24730, 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:40.272 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 156.89873242900001, 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 236.93816934899999 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdbcc0b6700 msgr2=0x7fdbcc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdbcc0b6700 0x7fdbcc0b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009580 tx=0x7fdbd0009c00 comp rx=0 tx=0).stop 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe410ff10 msgr2=0x7fdbe41a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:40.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe410ff10 0x7fdbe41a1cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009fd0 tx=0x7fdbdc00ef10 comp rx=0 tx=0).stop 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 shutdown_connections 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdbcc0b6700 0x7fdbcc0b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbe410ff10 0x7fdbe41a1cf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbe4076100 0x7fdbe419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 --2- 172.21.15.27:0/3810941482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbe40757b0 0x7fdbe419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 >> 172.21.15.27:0/3810941482 conn(0x7fdbe40ff440 msgr2=0x7fdbe4109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 shutdown_connections 2024-09-21T01:22:40.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:40.268+0000 7fdbea000640 1 -- 172.21.15.27:0/3810941482 wait complete. 2024-09-21T01:22:41.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T01:22:41.530 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:41.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:41 smithi027 ceph-mon[30375]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:22:42.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:41 smithi089 ceph-mon[34486]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:22:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:41 smithi098 ceph-mon[34372]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:22:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 -- 172.21.15.27:0/2827830537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f6010c020 msgr2=0x7f4f60112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 --2- 172.21.15.27:0/2827830537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f6010c020 0x7f4f60112850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009900 tx=0x7f4f5802a410 comp rx=0 tx=0).stop 2024-09-21T01:22:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 -- 172.21.15.27:0/2827830537 shutdown_connections 2024-09-21T01:22:42.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 --2- 172.21.15.27:0/2827830537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f6010c020 0x7f4f60112850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 --2- 172.21.15.27:0/2827830537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f6010b6d0 0x7f4f6010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 --2- 172.21.15.27:0/2827830537 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f60105d90 0x7f4f601061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 -- 172.21.15.27:0/2827830537 >> 172.21.15.27:0/2827830537 conn(0x7f4f60101860 msgr2=0x7f4f60103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.199+0000 7f4f67600640 1 -- 172.21.15.27:0/2827830537 shutdown_connections 2024-09-21T01:22:42.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.200+0000 7f4f67600640 1 -- 172.21.15.27:0/2827830537 wait complete. 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 Processor -- start 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 -- start start 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 0x7f4f6019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f6010b6d0 0x7f4f6019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f6010c020 0x7f4f601a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60115ce0 con 0x7f4f60105d90 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60115b60 con 0x7f4f6010b6d0 2024-09-21T01:22:42.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.201+0000 7f4f67600640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60115e60 con 0x7f4f6010c020 2024-09-21T01:22:42.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f64c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 0x7f4f6019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f64c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 0x7f4f6019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55584/0 (socket says 172.21.15.27:55584) 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f64c00640 1 -- 172.21.15.27:0/3626582340 learned_addr learned my addr 172.21.15.27:0/3626582340 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f6010b6d0 0x7f4f6019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f65600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f6010c020 0x7f4f601a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 -- 172.21.15.27:0/3626582340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f6010c020 msgr2=0x7f4f601a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f6010c020 0x7f4f601a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 -- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 msgr2=0x7f4f6019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 0x7f4f6019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 -- 172.21.15.27:0/3626582340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f601a25c0 con 0x7f4f6010b6d0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f64c00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 0x7f4f6019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.202+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f6010b6d0 0x7f4f6019cbf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f6006ca80 tx=0x7f4f4c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.203+0000 7f4f5d600640 1 -- 172.21.15.27:0/3626582340 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00cc40 con 0x7f4f6010b6d0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.203+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f601a28b0 con 0x7f4f6010b6d0 2024-09-21T01:22:42.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.203+0000 7f4f5d600640 1 -- 172.21.15.27:0/3626582340 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4c00cde0 con 0x7f4f6010b6d0 2024-09-21T01:22:42.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.203+0000 7f4f5d600640 1 -- 172.21.15.27:0/3626582340 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c018760 con 0x7f4f6010b6d0 2024-09-21T01:22:42.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.203+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f601a9010 con 0x7f4f6010b6d0 2024-09-21T01:22:42.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.206+0000 7f4f6e1d0740 1 Processor -- start 2024-09-21T01:22:42.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.206+0000 7f4f6e1d0740 1 -- start start 2024-09-21T01:22:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.206+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e59f60 0x555bb7200490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.206+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb72009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.206+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e39280 0x555bb7200f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bb7177d50 con 0x555bb6e59f60 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555bb7167180 con 0x555bb6e310a0 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bb716af90 con 0x555bb6e39280 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f65600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e59f60 0x555bb7200490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f5d600640 1 -- 172.21.15.27:0/3626582340 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f4f4c010a90 con 0x7f4f6010b6d0 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f64c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb72009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f65600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e59f60 0x555bb7200490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55586/0 (socket says 172.21.15.27:55586) 2024-09-21T01:22:42.209 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f65600640 1 -- 172.21.15.27:0/1013695894 learned_addr learned my addr 172.21.15.27:0/1013695894 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f64c00640 1 -- 172.21.15.27:0/1013695894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e39280 msgr2=0x555bb7200f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e39280 0x555bb7200f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f64c00640 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e39280 0x555bb7200f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f64c00640 1 -- 172.21.15.27:0/1013695894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e59f60 msgr2=0x555bb7200490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f64c00640 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e59f60 0x555bb7200490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.207+0000 7f4f64c00640 1 -- 172.21.15.27:0/1013695894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb7131a70 con 0x555bb6e310a0 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.208+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e39280 0x555bb7200f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:22:42.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.208+0000 7f4f64c00640 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb72009d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f54011870 tx=0x7f4f54011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70e5dc52425a27e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.208+0000 7f4f51c00640 1 -- 172.21.15.27:0/1013695894 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54011230 con 0x555bb6e310a0 2024-09-21T01:22:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.208+0000 7f4f51c00640 1 -- 172.21.15.27:0/1013695894 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f5400f9a0 con 0x555bb6e310a0 2024-09-21T01:22:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.208+0000 7f4f51c00640 1 -- 172.21.15.27:0/1013695894 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f540043d0 con 0x555bb6e310a0 2024-09-21T01:22:42.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.209+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/1013695894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 msgr2=0x555bb72009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.209+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb72009d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f54011870 tx=0x7f4f54011d30 comp rx=0 tx=0).stop 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/1013695894 shutdown_connections 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e39280 0x555bb7200f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb72009d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/1013695894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e59f60 0x555bb7200490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/1013695894 >> 172.21.15.27:0/1013695894 conn(0x555bb6e4e350 msgr2=0x555bb71fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/1013695894 shutdown_connections 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/1013695894 wait complete. 2024-09-21T01:22:42.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f5d600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4f480b6590 0x7f4f480b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f5d600640 1 -- 172.21.15.27:0/3626582340 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4f4c0e7d60 con 0x7f4f6010b6d0 2024-09-21T01:22:42.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.210+0000 7f4f64c00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4f480b6590 0x7f4f480b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 -- start start 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 Processor -- start 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f64c00640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4f480b6590 0x7f4f480b8aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f54011840 tx=0x7f4f54003cd0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb720c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e59f60 0x555bb7217f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bb6fc85f0 con 0x555bb6e39280 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555bb7102460 con 0x555bb6e310a0 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.211+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bb720ec70 con 0x555bb6e59f60 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f6e1d0740 10 client.0 _ll_register_callbacks cb 0x555bb7112210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f64c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb720c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55588/0 (socket says 172.21.15.27:55588) 2024-09-21T01:22:42.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 -- 172.21.15.27:0/80743299 learned_addr learned my addr 172.21.15.27:0/80743299 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:42.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f65600640 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e59f60 0x555bb7217f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 -- 172.21.15.27:0/80743299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e59f60 msgr2=0x555bb7217f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e59f60 0x555bb7217f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 -- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 msgr2=0x555bb720c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb720c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.212+0000 7f4f5fe00640 1 -- 172.21.15.27:0/80743299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb6ec5540 con 0x555bb6e39280 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c6a0 tx=0x7f4f4c0b7130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d204913d0bbb17e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f46000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:22:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f46000640 1 -- 172.21.15.27:0/80743299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0ad1e0 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f46000640 1 -- 172.21.15.27:0/80743299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4f4c0b3450 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bb7104a70 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f46000640 1 -- 172.21.15.27:0/80743299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f4c0ad000 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f6e1d0740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.213+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555bb711f6f0 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f46000640 1 -- 172.21.15.27:0/80743299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555bb7104a70 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f46000640 1 -- 172.21.15.27:0/80743299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4f4c0b3670 con 0x555bb6e39280 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f6e1d0740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f6e1d0740 10 client.15549 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f6e1d0740 20 client.15549 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6e188e0 0x555bb721e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f6e1d0740 4 client.15549 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 --> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb7179ca0 con 0x555bb6e188e0 2024-09-21T01:22:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.214+0000 7f4f64c00640 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6e188e0 0x555bb721e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.215+0000 7f4f64c00640 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6e188e0 0x555bb721e460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=126085c1b45902ef server_cookie=ccd6d288a3d8178f in_seq=0 out_seq=0 2024-09-21T01:22:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.215+0000 7f4f46000640 10 client.15549 ms_handle_connect on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f46000640 1 -- 172.21.15.27:0/80743299 <== mds.2 v2:172.21.15.89:6836/3556269004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bb7179ca0 con 0x555bb6e188e0 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f46000640 10 client.15549 handle_command_reply: tid=0 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 client.15549 shutdown 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 msgr2=0x555bb720c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00c6a0 tx=0x7f4f4c0b7130 comp rx=0 tx=0).stop 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 shutdown_connections 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6e188e0 0x555bb721e460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e59f60 0x555bb7217f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c560 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/80743299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e310a0 0x555bb720c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 >> 172.21.15.27:0/80743299 conn(0x555bb6e4e350 msgr2=0x555bb7203670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.216+0000 7f4f46000640 0 client.15549 ms_handle_reset on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.217+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 shutdown_connections 2024-09-21T01:22:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.217+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/80743299 wait complete. 2024-09-21T01:22:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.217+0000 7f4f6e1d0740 20 client.15549 trim_cache size 0 max 16384 2024-09-21T01:22:42.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 Processor -- start 2024-09-21T01:22:42.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 -- start start 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb720c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7207010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bb721c7a0 con 0x555bb6e39280 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555bb721ce70 con 0x555bb6e59f60 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bb721d540 con 0x555bb6e310a0 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55596/0 (socket says 172.21.15.27:55596) 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f5fe00640 1 -- 172.21.15.27:0/2028887276 learned_addr learned my addr 172.21.15.27:0/2028887276 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.236+0000 7f4f65600640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7207010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f64c00640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb720c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f5fe00640 1 -- 172.21.15.27:0/2028887276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 msgr2=0x555bb720c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb720c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f5fe00640 1 -- 172.21.15.27:0/2028887276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 msgr2=0x555bb7207010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7207010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f5fe00640 1 -- 172.21.15.27:0/2028887276 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb720e600 con 0x555bb6e39280 2024-09-21T01:22:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f64c00640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb720c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:22:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f65600640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7207010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0e8050 tx=0x7f4f4c0b21d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a3fcb023b8d476 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.237+0000 7f4f51200640 1 -- 172.21.15.27:0/2028887276 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0ad1e0 con 0x555bb6e39280 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f51200640 1 -- 172.21.15.27:0/2028887276 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4c0ad3a0 con 0x555bb6e39280 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f51200640 1 -- 172.21.15.27:0/2028887276 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0b6a30 con 0x555bb6e39280 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/2028887276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 msgr2=0x555bb720c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0e8050 tx=0x7f4f4c0b21d0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/2028887276 shutdown_connections 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7207010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb720c540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/2028887276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb720c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/2028887276 >> 172.21.15.27:0/2028887276 conn(0x555bb6e4e350 msgr2=0x555bb72180e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:42.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.238+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/2028887276 shutdown_connections 2024-09-21T01:22:42.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.239+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/2028887276 wait complete. 2024-09-21T01:22:42.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.239+0000 7f4f6e1d0740 1 -- start start 2024-09-21T01:22:42.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.239+0000 7f4f6e1d0740 1 Processor -- start 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb7227080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb7220530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7220c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bb721e2e0 con 0x555bb6e39280 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555bb721dc10 con 0x555bb6e59f60 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555bb721e9b0 con 0x555bb6e310a0 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f6e1d0740 10 client.0 _ll_register_callbacks cb 0x7f4f4c0a9ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb7220530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f64c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb7227080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb7220530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55606/0 (socket says 172.21.15.27:55606) 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f64c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb7227080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49912/0 (socket says 172.21.15.27:49912) 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f5fe00640 1 -- 172.21.15.27:0/3128711004 learned_addr learned my addr 172.21.15.27:0/3128711004 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.240+0000 7f4f65600640 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7220c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f64c00640 1 -- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 msgr2=0x555bb7220c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f64c00640 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7220c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f64c00640 1 -- 172.21.15.27:0/3128711004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 msgr2=0x555bb7220530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f64c00640 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb7220530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f64c00640 1 -- 172.21.15.27:0/3128711004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb7225cc0 con 0x555bb6e310a0 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f64c00640 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb7227080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f54011600 tx=0x7f4f54009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2be1ccc0bb0c6174 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f46000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f46000640 1 -- 172.21.15.27:0/3128711004 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400fd70 con 0x555bb6e310a0 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c0ad3a0 con 0x555bb6e310a0 2024-09-21T01:22:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.241+0000 7f4f46000640 1 -- 172.21.15.27:0/3128711004 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4f54005650 con 0x555bb6e310a0 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f46000640 1 -- 172.21.15.27:0/3128711004 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f54005850 con 0x555bb6e310a0 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 10 client.25579 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555bb7289aa0 con 0x555bb6e310a0 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f46000640 1 -- 172.21.15.27:0/3128711004 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4c0ad3a0 con 0x555bb6e310a0 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f46000640 1 -- 172.21.15.27:0/3128711004 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4f54039070 con 0x555bb6e310a0 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 10 client.25579 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 10 client.25579 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 20 client.25579 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6dedc90 0x555bb728d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 4 client.25579 mds_command: new command op to 24734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.242+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 --> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] -- command(tid 0: {"prefix": "status"}) -- 0x555bb7175750 con 0x555bb6dedc90 2024-09-21T01:22:42.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.243+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6dedc90 0x555bb728d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.243+0000 7f4f5fe00640 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6dedc90 0x555bb728d1a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4f8f16a9381feb2 server_cookie=75729b3b9892126a in_seq=0 out_seq=0 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.243+0000 7f4f46000640 10 client.25579 ms_handle_connect on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.243+0000 7f4f46000640 1 -- 172.21.15.27:0/3128711004 <== mds.2 v2:172.21.15.89:6836/3556269004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555bb7175750 con 0x555bb6dedc90 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.243+0000 7f4f46000640 10 client.25579 handle_command_reply: tid=0 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.243+0000 7f4f6e1d0740 1 client.25579 shutdown 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 msgr2=0x555bb7227080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb7227080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f54011600 tx=0x7f4f54009500 comp rx=0 tx=0).stop 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 shutdown_connections 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:6836/3556269004,v1:172.21.15.89:6837/3556269004] conn(0x555bb6dedc90 0x555bb728d1a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555bb6e59f60 0x555bb7220c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bb6e39280 0x555bb7220530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f46000640 0 client.25579 ms_handle_reset on v2:172.21.15.89:6836/3556269004 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 --2- 172.21.15.27:0/3128711004 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555bb6e310a0 0x555bb7227080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 >> 172.21.15.27:0/3128711004 conn(0x555bb6e4e350 msgr2=0x555bb7207010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:42.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 shutdown_connections 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 1 -- 172.21.15.27:0/3128711004 wait complete. 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.244+0000 7f4f6e1d0740 20 client.25579 trim_cache size 0 max 16384 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24734, 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 159.88689874600001, 2024-09-21T01:22:42.249 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-09-21T01:22:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T01:22:42.250 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 241.56321949299999 2024-09-21T01:22:42.250 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4f480b6590 msgr2=0x7f4f480b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4f480b6590 0x7f4f480b8aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f54011840 tx=0x7f4f54003cd0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f6010b6d0 msgr2=0x7f4f6019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f6010b6d0 0x7f4f6019cbf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4f6006ca80 tx=0x7f4f4c00ee50 comp rx=0 tx=0).stop 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 shutdown_connections 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4f480b6590 0x7f4f480b8aa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f6010c020 0x7f4f601a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4f6010b6d0 0x7f4f6019cbf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 --2- 172.21.15.27:0/3626582340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f60105d90 0x7f4f6019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 >> 172.21.15.27:0/3626582340 conn(0x7f4f60101860 msgr2=0x7f4f6010ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 shutdown_connections 2024-09-21T01:22:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:42.245+0000 7f4f67600640 1 -- 172.21.15.27:0/3626582340 wait complete. 2024-09-21T01:22:43.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.g status 2024-09-21T01:22:43.562 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:43.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:43 smithi089 ceph-mon[34486]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:22:44.052 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:43 smithi027 ceph-mon[30375]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:22:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:43 smithi098 ceph-mon[34372]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 -- 172.21.15.27:0/2370169434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1039b0 msgr2=0x7fb0cc103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/2370169434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1039b0 0x7fb0cc103dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009900 tx=0x7fb0bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 -- 172.21.15.27:0/2370169434 shutdown_connections 2024-09-21T01:22:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/2370169434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0cc109c40 0x7fb0cc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/2370169434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/2370169434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1039b0 0x7fb0cc103dc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 -- 172.21.15.27:0/2370169434 >> 172.21.15.27:0/2370169434 conn(0x7fb0cc0ff440 msgr2=0x7fb0cc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.237+0000 7fb0d0e00640 1 -- 172.21.15.27:0/2370169434 shutdown_connections 2024-09-21T01:22:44.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.238+0000 7fb0d0e00640 1 -- 172.21.15.27:0/2370169434 wait complete. 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 Processor -- start 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 -- start start 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc1039b0 0x7fb0cc19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0cc109c40 0x7fb0cc1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc113950 con 0x7fb0cc1092f0 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc1137d0 con 0x7fb0cc109c40 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0d0e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc113ad0 con 0x7fb0cc1039b0 2024-09-21T01:22:44.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.239+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55624/0 (socket says 172.21.15.27:55624) 2024-09-21T01:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1218854981 learned_addr learned my addr 172.21.15.27:0/1218854981 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:44.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc1039b0 0x7fb0cc19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0cc109c40 0x7fb0cc1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1218854981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc1039b0 msgr2=0x7fb0cc19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc1039b0 0x7fb0cc19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1218854981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0cc109c40 msgr2=0x7fb0cc1a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0cc109c40 0x7fb0cc1a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1218854981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc1a2470 con 0x7fb0cc1092f0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.240+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc19caa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400d740 tx=0x7fb0b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.241+0000 7fb0c8c00640 1 -- 172.21.15.27:0/1218854981 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4017070 con 0x7fb0cc1092f0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.241+0000 7fb0c8c00640 1 -- 172.21.15.27:0/1218854981 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b400de50 con 0x7fb0cc1092f0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.241+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc1a2760 con 0x7fb0cc1092f0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.241+0000 7fb0c8c00640 1 -- 172.21.15.27:0/1218854981 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400f620 con 0x7fb0cc1092f0 2024-09-21T01:22:44.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.241+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1a8ec0 con 0x7fb0cc1092f0 2024-09-21T01:22:44.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.245+0000 7fb0d9084740 1 Processor -- start 2024-09-21T01:22:44.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.245+0000 7fb0d9084740 1 -- start start 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115cebc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cebcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cebd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0d9084740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56115ce33d50 con 0x56115cb15f60 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0d9084740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56115ce23180 con 0x56115caed0a0 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0d9084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56115ce26f90 con 0x56115caf5280 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115cebc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0c8c00640 1 -- 172.21.15.27:0/1218854981 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7fb0b401a070 con 0x7fb0cc1092f0 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115cebc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55626/0 (socket says 172.21.15.27:55626) 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0cbe00640 1 -- 172.21.15.27:0/1702592886 learned_addr learned my addr 172.21.15.27:0/1702592886 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:44.248 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.246+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cebcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cebd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0cb400640 1 -- 172.21.15.27:0/1702592886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 msgr2=0x56115cebd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cebd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0cb400640 1 -- 172.21.15.27:0/1702592886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 msgr2=0x56115cebc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115cebc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0cb400640 1 -- 172.21.15.27:0/1702592886 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115cdeda70 con 0x56115caed0a0 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cebd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.247+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cebcbb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc02a9e0 tx=0x7fb0bc004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b7dc00a6bc816f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.248+0000 7fb0c0c00640 1 -- 172.21.15.27:0/1702592886 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc035040 con 0x56115caed0a0 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.248+0000 7fb0c0c00640 1 -- 172.21.15.27:0/1702592886 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc0054f0 con 0x56115caed0a0 2024-09-21T01:22:44.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.248+0000 7fb0c0c00640 1 -- 172.21.15.27:0/1702592886 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0035f0 con 0x56115caed0a0 2024-09-21T01:22:44.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.248+0000 7fb0c8c00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb0b00b6680 0x7fb0b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0c8c00640 1 -- 172.21.15.27:0/1218854981 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb0b40e7450 con 0x7fb0cc1092f0 2024-09-21T01:22:44.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 -- 172.21.15.27:0/1702592886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 msgr2=0x56115cebcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cebcbb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc02a9e0 tx=0x7fb0bc004170 comp rx=0 tx=0).stop 2024-09-21T01:22:44.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb0b00b6680 0x7fb0b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 -- 172.21.15.27:0/1702592886 shutdown_connections 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cebd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cebcbb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1702592886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115cebc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 -- 172.21.15.27:0/1702592886 >> 172.21.15.27:0/1702592886 conn(0x56115cb0a350 msgr2=0x56115ceba560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 -- 172.21.15.27:0/1702592886 shutdown_connections 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0d9084740 1 -- 172.21.15.27:0/1702592886 wait complete. 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.249+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb0b00b6680 0x7fb0b00b8b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4005fd0 tx=0x7fb0c4009500 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 -- start start 2024-09-21T01:22:44.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 Processor -- start 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cec82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cec87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115ced40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56115cc845f0 con 0x56115cb15f60 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56115cdbe460 con 0x56115caed0a0 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.250+0000 7fb0d9084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56115cecadf0 con 0x56115caf5280 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0d9084740 10 client.0 _ll_register_callbacks cb 0x56115cdce210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cec87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115ced40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0cb400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cec82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cec87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:49956/0 (socket says 172.21.15.27:49956) 2024-09-21T01:22:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1988968252 learned_addr learned my addr 172.21.15.27:0/1988968252 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1988968252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 msgr2=0x56115cec82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cec82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 msgr2=0x56115ced40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115ced40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.251+0000 7fb0c3e00640 1 -- 172.21.15.27:0/1988968252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115cb81540 con 0x56115caf5280 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cec87e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40acf10 tx=0x7fb0b40ace80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46cf6a329b4dd954 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0b9600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0b9600640 1 -- 172.21.15.27:0/1988968252 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4021030 con 0x56115caf5280 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0b9600640 1 -- 172.21.15.27:0/1988968252 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb0b40a88d0 con 0x56115caf5280 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56115cdc0a70 con 0x56115caf5280 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0b9600640 1 -- 172.21.15.27:0/1988968252 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0b40e4c60 con 0x56115caf5280 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0d9084740 10 client.25588 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56115cddb6f0 con 0x56115caf5280 2024-09-21T01:22:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.252+0000 7fb0b9600640 1 -- 172.21.15.27:0/1988968252 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56115cdc0a70 con 0x56115caf5280 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0b9600640 1 -- 172.21.15.27:0/1988968252 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb0b40abc50 con 0x56115caf5280 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0d9084740 10 client.25588 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0d9084740 10 client.25588 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0d9084740 20 client.25588 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115cad48e0 0x56115ceda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0d9084740 4 client.25588 mds_command: new command op to 24752 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 --> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115ce35ca0 con 0x56115cad48e0 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.253+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115cad48e0 0x56115ceda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.254+0000 7fb0cb400640 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115cad48e0 0x56115ceda5d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=581b02dd4a478d80 server_cookie=1ec5a55c88c0c51a in_seq=0 out_seq=0 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.254+0000 7fb0b9600640 10 client.25588 ms_handle_connect on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.254+0000 7fb0b9600640 1 -- 172.21.15.27:0/1988968252 <== mds.1 v2:172.21.15.27:6838/3115497172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56115ce35ca0 con 0x56115cad48e0 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.254+0000 7fb0b9600640 10 client.25588 handle_command_reply: tid=0 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 client.25588 shutdown 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 msgr2=0x56115cec87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cec87e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40acf10 tx=0x7fb0b40ace80 comp rx=0 tx=0).stop 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 shutdown_connections 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115cad48e0 0x56115ceda5d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115cb15f60 0x56115ced40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115caf5280 0x56115cec87e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0b9600640 0 client.25588 ms_handle_reset on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 --2- 172.21.15.27:0/1988968252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caed0a0 0x56115cec82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 >> 172.21.15.27:0/1988968252 conn(0x56115cb0a350 msgr2=0x56115cebf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:44.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 shutdown_connections 2024-09-21T01:22:44.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 1 -- 172.21.15.27:0/1988968252 wait complete. 2024-09-21T01:22:44.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.255+0000 7fb0d9084740 20 client.25588 trim_cache size 0 max 16384 2024-09-21T01:22:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 Processor -- start 2024-09-21T01:22:44.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 -- start start 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115ced9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cec3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cec3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56115ced8910 con 0x56115caed0a0 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56115ced8fe0 con 0x56115caf5280 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0d9084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56115ced96b0 con 0x56115cb15f60 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0cb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115ced9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0cb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115ced9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55648/0 (socket says 172.21.15.27:55648) 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0cb400640 1 -- 172.21.15.27:0/2812511090 learned_addr learned my addr 172.21.15.27:0/2812511090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cec3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.275+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cec3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0c3e00640 1 -- 172.21.15.27:0/2812511090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 msgr2=0x56115cec3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cec3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0c3e00640 1 -- 172.21.15.27:0/2812511090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 msgr2=0x56115ced9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115ced9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0c3e00640 1 -- 172.21.15.27:0/2812511090 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115cebe960 con 0x56115caf5280 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cec3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cec3250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009610 tx=0x7fb0b40a95d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd37d7fac5acaed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0bbe00640 1 -- 172.21.15.27:0/2812511090 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4021070 con 0x56115caf5280 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0bbe00640 1 -- 172.21.15.27:0/2812511090 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b40b19b0 con 0x56115caf5280 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.276+0000 7fb0bbe00640 1 -- 172.21.15.27:0/2812511090 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4098040 con 0x56115caf5280 2024-09-21T01:22:44.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 -- 172.21.15.27:0/2812511090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 msgr2=0x56115cec3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cec3250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009610 tx=0x7fb0b40a95d0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 -- 172.21.15.27:0/2812511090 shutdown_connections 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cec3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cec3250 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 --2- 172.21.15.27:0/2812511090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115ced9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 -- 172.21.15.27:0/2812511090 >> 172.21.15.27:0/2812511090 conn(0x56115cb0a350 msgr2=0x56115cec8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 -- 172.21.15.27:0/2812511090 shutdown_connections 2024-09-21T01:22:44.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.277+0000 7fb0d9084740 1 -- 172.21.15.27:0/2812511090 wait complete. 2024-09-21T01:22:44.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 -- start start 2024-09-21T01:22:44.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 Processor -- start 2024-09-21T01:22:44.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115cedc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cedcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cedd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56115ceda450 con 0x56115caed0a0 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56115ced9d80 con 0x56115caf5280 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.278+0000 7fb0d9084740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56115cedab20 con 0x56115cb15f60 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0d9084740 10 client.0 _ll_register_callbacks cb 0x7fb0b4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cedcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cedcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42276/0 (socket says 172.21.15.27:42276) 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 -- 172.21.15.27:0/53462801 learned_addr learned my addr 172.21.15.27:0/53462801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0cb400640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115cedc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cedd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 -- 172.21.15.27:0/53462801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 msgr2=0x56115cedd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cedd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 -- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 msgr2=0x56115cedc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115cedc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 -- 172.21.15.27:0/53462801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115cee1f30 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0cbe00640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cedd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0c3e00640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cedcce0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40a9550 tx=0x7fb0b40ac5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958f5fcb99793847 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.279+0000 7fb0b9600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0b9600640 1 -- 172.21.15.27:0/53462801 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4021070 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0b9600640 1 -- 172.21.15.27:0/53462801 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb0b40abc50 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b40b19b0 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0b9600640 1 -- 172.21.15.27:0/53462801 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0b40d4040 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0d9084740 10 client.25613 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56115cf45cb0 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0b9600640 1 -- 172.21.15.27:0/53462801 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b40b19b0 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0b9600640 1 -- 172.21.15.27:0/53462801 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb0b40d41e0 con 0x56115caf5280 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0d9084740 10 client.25613 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.280+0000 7fb0d9084740 10 client.25613 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-21T01:22:44.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0d9084740 20 client.25613 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:44.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0d9084740 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115caa9c90 0x56115cf48fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:44.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0d9084740 4 client.25613 mds_command: new command op to 24752 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:44.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 --> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] -- command(tid 0: {"prefix": "status"}) -- 0x56115ce31750 con 0x56115caa9c90 2024-09-21T01:22:44.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0cb400640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115caa9c90 0x56115cf48fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0cb400640 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115caa9c90 0x56115cf48fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=346137fe63c22418 server_cookie=7dbae26479e0a7c3 in_seq=0 out_seq=0 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.281+0000 7fb0b9600640 10 client.25613 ms_handle_connect on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0b9600640 1 -- 172.21.15.27:0/53462801 <== mds.1 v2:172.21.15.27:6838/3115497172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56115ce31750 con 0x56115caa9c90 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0b9600640 10 client.25613 handle_command_reply: tid=0 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 client.25613 shutdown 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 msgr2=0x56115cedcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cedcce0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40a9550 tx=0x7fb0b40ac5b0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 shutdown_connections 2024-09-21T01:22:44.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:6838/3115497172,v1:172.21.15.27:6839/3115497172] conn(0x56115caa9c90 0x56115cf48fb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56115cb15f60 0x56115cedd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56115caf5280 0x56115cedcce0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0b9600640 0 client.25613 ms_handle_reset on v2:172.21.15.27:6838/3115497172 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 --2- 172.21.15.27:0/53462801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56115caed0a0 0x56115cedc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 >> 172.21.15.27:0/53462801 conn(0x56115cb0a350 msgr2=0x56115cec68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 shutdown_connections 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 1 -- 172.21.15.27:0/53462801 wait complete. 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.282+0000 7fb0d9084740 20 client.25613 trim_cache size 0 max 16384 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:44.287 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24752, 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 162.93489426400001, 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 236.870253593 2024-09-21T01:22:44.288 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb0b00b6680 msgr2=0x7fb0b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb0b00b6680 0x7fb0b00b8b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4005fd0 tx=0x7fb0c4009500 comp rx=0 tx=0).stop 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 msgr2=0x7fb0cc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc19caa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400d740 tx=0x7fb0b400dc00 comp rx=0 tx=0).stop 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 shutdown_connections 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fb0b00b6680 0x7fb0b00b8b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0cc109c40 0x7fb0cc1a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0cc1092f0 0x7fb0cc19caa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 --2- 172.21.15.27:0/1218854981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc1039b0 0x7fb0cc19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:44.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 >> 172.21.15.27:0/1218854981 conn(0x7fb0cc0ff440 msgr2=0x7fb0cc108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:44.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.283+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 shutdown_connections 2024-09-21T01:22:44.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:44.284+0000 7fb0d0e00640 1 -- 172.21.15.27:0/1218854981 wait complete. 2024-09-21T01:22:45.222 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T01:22:45.618 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:22:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:45 smithi089 ceph-mon[34486]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:22:46.005 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:45 smithi027 ceph-mon[30375]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:22:46.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:45 smithi098 ceph-mon[34372]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 -- 172.21.15.27:0/1402036675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 msgr2=0x7f664c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 --2- 172.21.15.27:0/1402036675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c109700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f663802a400 comp rx=0 tx=0).stop 2024-09-21T01:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 -- 172.21.15.27:0/1402036675 shutdown_connections 2024-09-21T01:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 --2- 172.21.15.27:0/1402036675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f664c109c40 0x7f664c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 --2- 172.21.15.27:0/1402036675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c109700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 --2- 172.21.15.27:0/1402036675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 -- 172.21.15.27:0/1402036675 >> 172.21.15.27:0/1402036675 conn(0x7f664c0ff400 msgr2=0x7f664c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 -- 172.21.15.27:0/1402036675 shutdown_connections 2024-09-21T01:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.307+0000 7f664be00640 1 -- 172.21.15.27:0/1402036675 wait complete. 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.308+0000 7f664be00640 1 Processor -- start 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.308+0000 7f664be00640 1 -- start start 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f664c109c40 0x7f664c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f664c113900 con 0x7f664c109c40 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f664c113780 con 0x7f664c1039b0 2024-09-21T01:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f664c113a80 con 0x7f664c1092f0 2024-09-21T01:22:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f664c109c40 0x7f664c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40162/0 (socket says 172.21.15.27:40162) 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:44530/0 (socket says 172.21.15.27:44530) 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.309+0000 7f664aa00640 1 -- 172.21.15.27:0/544152674 learned_addr learned my addr 172.21.15.27:0/544152674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664aa00640 1 -- 172.21.15.27:0/544152674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 msgr2=0x7f664c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664aa00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664aa00640 1 -- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f664c109c40 msgr2=0x7f664c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664aa00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f664c109c40 0x7f664c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664aa00640 1 -- 172.21.15.27:0/544152674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664c1a8570 con 0x7f664c1039b0 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664a000640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.310+0000 7f664aa00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f663c00e9f0 tx=0x7f663c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.311+0000 7f6650a00640 1 -- 172.21.15.27:0/544152674 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c00cd80 con 0x7f664c1039b0 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.311+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664c1a8860 con 0x7f664c1039b0 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.311+0000 7f6650a00640 1 -- 172.21.15.27:0/544152674 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c0108e0 con 0x7f664c1039b0 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.311+0000 7f6650a00640 1 -- 172.21.15.27:0/544152674 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c018980 con 0x7f664c1039b0 2024-09-21T01:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.311+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664c1a8d70 con 0x7f664c1039b0 2024-09-21T01:22:46.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.315+0000 7f6658a34740 1 Processor -- start 2024-09-21T01:22:46.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.315+0000 7f6658a34740 1 -- start start 2024-09-21T01:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.315+0000 7f6658a34740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 0x55770d814670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.315+0000 7f6658a34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d814bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f6658a34740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d44d280 0x55770d8150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f6658a34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55770d78bd50 con 0x55770d4450a0 2024-09-21T01:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f6658a34740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55770d77b180 con 0x55770d46df60 2024-09-21T01:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f6658a34740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55770d77ef90 con 0x55770d44d280 2024-09-21T01:22:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 0x55770d814670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664b400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 0x55770d814670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:40178/0 (socket says 172.21.15.27:40178) 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664b400640 1 -- 172.21.15.27:0/3976178015 learned_addr learned my addr 172.21.15.27:0/3976178015 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664a000640 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d44d280 0x55770d8150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f6650a00640 1 -- 172.21.15.27:0/544152674 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f663c010430 con 0x7f664c1039b0 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664b400640 1 -- 172.21.15.27:0/3976178015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d44d280 msgr2=0x55770d8150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664b400640 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d44d280 0x55770d8150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.316+0000 7f664b400640 1 -- 172.21.15.27:0/3976178015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 msgr2=0x55770d814bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f664b400640 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d814bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f664b400640 1 -- 172.21.15.27:0/3976178015 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770d745a70 con 0x55770d46df60 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f664a000640 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d44d280 0x55770d8150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f664b400640 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 0x55770d814670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6644009fd0 tx=0x7f66440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9069c183dcc2181 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f6637e00640 1 -- 172.21.15.27:0/3976178015 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644004d20 con 0x55770d46df60 2024-09-21T01:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f6637e00640 1 -- 172.21.15.27:0/3976178015 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6644004050 con 0x55770d46df60 2024-09-21T01:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.317+0000 7f6637e00640 1 -- 172.21.15.27:0/3976178015 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644005280 con 0x55770d46df60 2024-09-21T01:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.318+0000 7f6650a00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f66300b6680 0x7f66300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.318+0000 7f6658a34740 1 -- 172.21.15.27:0/3976178015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 msgr2=0x55770d814670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.318+0000 7f6658a34740 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 0x55770d814670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6644009fd0 tx=0x7f66440118d0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6650a00640 1 -- 172.21.15.27:0/544152674 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f663c0e7b80 con 0x7f664c1039b0 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 -- 172.21.15.27:0/3976178015 shutdown_connections 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d44d280 0x55770d8150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d814bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 --2- 172.21.15.27:0/3976178015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d46df60 0x55770d814670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 -- 172.21.15.27:0/3976178015 >> 172.21.15.27:0/3976178015 conn(0x55770d462350 msgr2=0x55770d812560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f664aa00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f66300b6680 0x7f66300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 -- 172.21.15.27:0/3976178015 shutdown_connections 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.319+0000 7f6658a34740 1 -- 172.21.15.27:0/3976178015 wait complete. 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 -- start start 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f664aa00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f66300b6680 0x7f66300b8b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55770d8132f0 tx=0x7f663c009700 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 Processor -- start 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d4450a0 0x55770d820240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d820780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d46df60 0x55770d82c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55770d5dc5f0 con 0x55770d46df60 2024-09-21T01:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55770d716460 con 0x55770d44d280 2024-09-21T01:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.320+0000 7f6658a34740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55770d822ea0 con 0x55770d4450a0 2024-09-21T01:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f6658a34740 10 client.0 _ll_register_callbacks cb 0x55770d726210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:46.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d4450a0 0x55770d820240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664a000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d820780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664b400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d4450a0 0x55770d820240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:44550/0 (socket says 172.21.15.27:44550) 2024-09-21T01:22:46.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664b400640 1 -- 172.21.15.27:0/533466833 learned_addr learned my addr 172.21.15.27:0/533466833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:46.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664b400640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d46df60 0x55770d82c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664a000640 1 -- 172.21.15.27:0/533466833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d4450a0 msgr2=0x55770d820240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664a000640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d4450a0 0x55770d820240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664a000640 1 -- 172.21.15.27:0/533466833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d46df60 msgr2=0x55770d82c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664a000640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d46df60 0x55770d82c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664a000640 1 -- 172.21.15.27:0/533466833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770d4d9540 con 0x55770d44d280 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.321+0000 7f664b400640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d46df60 0x55770d82c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.322+0000 7f664a000640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d820780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f663802a920 tx=0x7f66380316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb71d85b50c997e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.322+0000 7f6634c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.322+0000 7f6634c00640 1 -- 172.21.15.27:0/533466833 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035040 con 0x55770d44d280 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.322+0000 7f6634c00640 1 -- 172.21.15.27:0/533466833 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6638004040 con 0x55770d44d280 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.322+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55770d718a70 con 0x55770d44d280 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.322+0000 7f6634c00640 1 -- 172.21.15.27:0/533466833 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6638004240 con 0x55770d44d280 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 10 client.25625 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55770d7336f0 con 0x55770d44d280 2024-09-21T01:22:46.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6634c00640 1 -- 172.21.15.27:0/533466833 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55770d718a70 con 0x55770d44d280 2024-09-21T01:22:46.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6634c00640 1 -- 172.21.15.27:0/533466833 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6638004af0 con 0x55770d44d280 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 10 client.25625 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 10 client.25625 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 20 client.25625 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d42c8e0 0x55770d832770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 4 client.25625 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.323+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55770d78dca0 con 0x55770d42c8e0 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.324+0000 7f664b400640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d42c8e0 0x55770d832770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.324+0000 7f664b400640 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d42c8e0 0x55770d832770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd6c3afba381399 server_cookie=e2b16de1d287e259 in_seq=0 out_seq=0 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.324+0000 7f6634c00640 10 client.25625 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6634c00640 1 -- 172.21.15.27:0/533466833 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55770d78dca0 con 0x55770d42c8e0 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6634c00640 10 client.25625 handle_command_reply: tid=0 2024-09-21T01:22:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 client.25625 shutdown 2024-09-21T01:22:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 msgr2=0x55770d820780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d820780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f663802a920 tx=0x7f66380316d0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 shutdown_connections 2024-09-21T01:22:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d46df60 0x55770d82c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d42c8e0 0x55770d832770 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d820780 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 --2- 172.21.15.27:0/533466833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d4450a0 0x55770d820240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6634c00640 0 client.25625 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.325+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 >> 172.21.15.27:0/533466833 conn(0x55770d462350 msgr2=0x55770d817800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.326+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 shutdown_connections 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.326+0000 7f6658a34740 1 -- 172.21.15.27:0/533466833 wait complete. 2024-09-21T01:22:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.326+0000 7f6658a34740 20 client.25625 trim_cache size 0 max 16384 2024-09-21T01:22:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 Processor -- start 2024-09-21T01:22:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 -- start start 2024-09-21T01:22:46.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d81b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d81b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d81e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55770d830ab0 con 0x55770d4450a0 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55770d831180 con 0x55770d44d280 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.345+0000 7f6658a34740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55770d831850 con 0x55770d46df60 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d81b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d81b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33426/0 (socket says 172.21.15.27:33426) 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 -- 172.21.15.27:0/4166315268 learned_addr learned my addr 172.21.15.27:0/4166315268 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664a000640 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d81b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d81e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 -- 172.21.15.27:0/4166315268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 msgr2=0x55770d81e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d81e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 -- 172.21.15.27:0/4166315268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 msgr2=0x55770d81b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d81b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.346+0000 7f664b400640 1 -- 172.21.15.27:0/4166315268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770d816990 con 0x55770d4450a0 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.347+0000 7f664b400640 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d81b1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6644005fd0 tx=0x7f6644000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3109b98726c06b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.347+0000 7f6637400640 1 -- 172.21.15.27:0/4166315268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644004cc0 con 0x55770d4450a0 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.347+0000 7f6637400640 1 -- 172.21.15.27:0/4166315268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6644004500 con 0x55770d4450a0 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.347+0000 7f6637400640 1 -- 172.21.15.27:0/4166315268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644005780 con 0x55770d4450a0 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.347+0000 7f6658a34740 1 -- 172.21.15.27:0/4166315268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 msgr2=0x55770d81b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.347+0000 7f6658a34740 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d81b1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6644005fd0 tx=0x7f6644000f40 comp rx=0 tx=0).stop 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 -- 172.21.15.27:0/4166315268 shutdown_connections 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d81e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d81b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 --2- 172.21.15.27:0/4166315268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d81b1a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 -- 172.21.15.27:0/4166315268 >> 172.21.15.27:0/4166315268 conn(0x55770d462350 msgr2=0x55770d820220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:46.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 -- 172.21.15.27:0/4166315268 shutdown_connections 2024-09-21T01:22:46.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 -- 172.21.15.27:0/4166315268 wait complete. 2024-09-21T01:22:46.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.348+0000 7f6658a34740 1 -- start start 2024-09-21T01:22:46.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 Processor -- start 2024-09-21T01:22:46.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d834b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d8350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d8355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55770d8325f0 con 0x55770d4450a0 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55770d831f20 con 0x55770d44d280 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55770d832cc0 con 0x55770d46df60 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f6658a34740 10 client.0 _ll_register_callbacks cb 0x7f6638005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f664b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d834b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.349+0000 7f664b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d834b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33428/0 (socket says 172.21.15.27:33428) 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664b400640 1 -- 172.21.15.27:0/2478172415 learned_addr learned my addr 172.21.15.27:0/2478172415 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d8350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664b400640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d8355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 -- 172.21.15.27:0/2478172415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 msgr2=0x55770d8355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d8355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 -- 172.21.15.27:0/2478172415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 msgr2=0x55770d834b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d834b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 -- 172.21.15.27:0/2478172415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770d83a4a0 con 0x55770d44d280 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664b400640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d834b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f664a000640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d8350a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6638003640 tx=0x7f6638005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898c717620aa86be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.350+0000 7f6634c00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6634c00640 1 -- 172.21.15.27:0/2478172415 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638058020 con 0x55770d44d280 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6644004500 con 0x55770d44d280 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6634c00640 1 -- 172.21.15.27:0/2478172415 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6638004770 con 0x55770d44d280 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6634c00640 1 -- 172.21.15.27:0/2478172415 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6638035040 con 0x55770d44d280 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6658a34740 10 client.25631 fetch_fsmap learned FSMap version 29 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55770d89e020 con 0x55770d44d280 2024-09-21T01:22:46.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6634c00640 1 -- 172.21.15.27:0/2478172415 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6644004500 con 0x55770d44d280 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6634c00640 1 -- 172.21.15.27:0/2478172415 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6638004fc0 con 0x55770d44d280 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6658a34740 10 client.25631 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6658a34740 10 client.25631 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.351+0000 7f6658a34740 20 client.25631 populate_metadata read hostname 'smithi027' 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.352+0000 7f6658a34740 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d401c90 0x55770d8a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.352+0000 7f6658a34740 4 client.25631 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.352+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "status"}) -- 0x55770d789750 con 0x55770d401c90 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.352+0000 7f664b400640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d401c90 0x55770d8a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.352+0000 7f664b400640 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d401c90 0x55770d8a1950 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471e24cefb8de062 server_cookie=39a8bb06027ae02d in_seq=0 out_seq=0 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.352+0000 7f6634c00640 10 client.25631 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6634c00640 1 -- 172.21.15.27:0/2478172415 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55770d789750 con 0x55770d401c90 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6634c00640 10 client.25631 handle_command_reply: tid=0 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 client.25631 shutdown 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 msgr2=0x55770d8350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d8350a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6638003640 tx=0x7f6638005850 comp rx=0 tx=0).stop 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 shutdown_connections 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55770d46df60 0x55770d8355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55770d401c90 0x55770d8a1950 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55770d44d280 0x55770d8350a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 --2- 172.21.15.27:0/2478172415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55770d4450a0 0x55770d834b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 >> 172.21.15.27:0/2478172415 conn(0x55770d462350 msgr2=0x55770d811df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6634c00640 0 client.25631 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 shutdown_connections 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.353+0000 7f6658a34740 1 -- 172.21.15.27:0/2478172415 wait complete. 2024-09-21T01:22:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f6658a34740 20 client.25631 trim_cache size 0 max 16384 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "fe42ec90-77b5-11ef-baf5-efdc52797490", 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 24776, 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 217.622887795, 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 29, 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 90, 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T01:22:46.357 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 232.13082787499999 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f66300b6680 msgr2=0x7f66300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f66300b6680 0x7f66300b8b90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55770d8132f0 tx=0x7f663c009700 comp rx=0 tx=0).stop 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 msgr2=0x7f664c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f663c00e9f0 tx=0x7f663c00eeb0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 shutdown_connections 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f66300b6680 0x7f66300b8b90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f664c109c40 0x7f664c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f664c1092f0 0x7f664c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 --2- 172.21.15.27:0/544152674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f664c1039b0 0x7f664c075710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:22:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 >> 172.21.15.27:0/544152674 conn(0x7f664c0ff400 msgr2=0x7f664c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:22:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 shutdown_connections 2024-09-21T01:22:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:22:46.354+0000 7f664be00640 1 -- 172.21.15.27:0/544152674 wait complete. 2024-09-21T01:22:47.280 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T01:22:47.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:47 smithi089 ceph-mon[34486]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T01:22:48.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:47 smithi098 ceph-mon[34372]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T01:22:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:47 smithi027 ceph-mon[30375]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T01:22:50.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:49 smithi098 ceph-mon[34372]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T01:22:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:49 smithi027 ceph-mon[30375]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T01:22:50.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:49 smithi089 ceph-mon[34486]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T01:22:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:51 smithi098 ceph-mon[34372]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T01:22:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:51 smithi027 ceph-mon[30375]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T01:22:52.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:51 smithi089 ceph-mon[34486]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T01:22:54.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:53 smithi098 ceph-mon[34372]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:22:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:53 smithi027 ceph-mon[30375]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:22:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:53 smithi089 ceph-mon[34486]: pgmap v414: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:22:56.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:55 smithi098 ceph-mon[34372]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-21T01:22:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:55 smithi027 ceph-mon[30375]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-21T01:22:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:55 smithi089 ceph-mon[34486]: pgmap v415: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-21T01:22:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:57 smithi098 ceph-mon[34372]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:22:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:57 smithi027 ceph-mon[30375]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:22:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:57 smithi089 ceph-mon[34486]: pgmap v416: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:23:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:22:59 smithi098 ceph-mon[34372]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:00.180 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:22:59 smithi027 ceph-mon[30375]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:00.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:22:59 smithi089 ceph-mon[34486]: pgmap v417: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:00.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:23:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:23:00.436+0000 7f6532600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:23:00.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:23:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:23:00.436+0000 7f6532600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:23:00.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:23:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:23:00.436+0000 7f6532600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:23:00.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:23:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:23:00.436+0000 7f6532600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:23:00.931 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:23:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:23:00.436+0000 7f6532600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T01:23:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:01 smithi098 ceph-mon[34372]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:23:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:01 smithi098 ceph-mon[34372]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-09-21T01:23:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:01 smithi027 ceph-mon[30375]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:23:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:01 smithi027 ceph-mon[30375]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-09-21T01:23:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:01 smithi089 ceph-mon[34486]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:23:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:01 smithi089 ceph-mon[34486]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-09-21T01:23:02.280 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T01:23:02.281 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:23:02.706 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:23:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 -- 172.21.15.27:0/3517405256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 msgr2=0x7f37f010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 --2- 172.21.15.27:0/3517405256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f010bae0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009900 tx=0x7f37dc02a410 comp rx=0 tx=0).stop 2024-09-21T01:23:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 -- 172.21.15.27:0/3517405256 shutdown_connections 2024-09-21T01:23:03.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 --2- 172.21.15.27:0/3517405256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f010c020 0x7f37f0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 --2- 172.21.15.27:0/3517405256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f010bae0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 --2- 172.21.15.27:0/3517405256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f0105d90 0x7f37f01061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.401+0000 7f37f6e00640 1 -- 172.21.15.27:0/3517405256 >> 172.21.15.27:0/3517405256 conn(0x7f37f0101860 msgr2=0x7f37f0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:03.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.403+0000 7f37f6e00640 1 -- 172.21.15.27:0/3517405256 shutdown_connections 2024-09-21T01:23:03.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.403+0000 7f37f6e00640 1 -- 172.21.15.27:0/3517405256 wait complete. 2024-09-21T01:23:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.404+0000 7f37f6e00640 1 Processor -- start 2024-09-21T01:23:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.404+0000 7f37f6e00640 1 -- start start 2024-09-21T01:23:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f6e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 0x7f37f019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f6e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:03.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f6e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f010c020 0x7f37f01a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f6e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0115ce0 con 0x7f37f010b6d0 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f6e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0115b60 con 0x7f37f0105d90 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f6e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0115e60 con 0x7f37f010c020 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 0x7f37f019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37efe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 0x7f37f019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57482/0 (socket says 172.21.15.27:57482) 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37efe00640 1 -- 172.21.15.27:0/1418838534 learned_addr learned my addr 172.21.15.27:0/1418838534 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37f4e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f010c020 0x7f37f01a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.405+0000 7f37ef400640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37efe00640 1 -- 172.21.15.27:0/1418838534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f010c020 msgr2=0x7f37f01a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37efe00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f010c020 0x7f37f01a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37efe00640 1 -- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 msgr2=0x7f37f019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37efe00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37efe00640 1 -- 172.21.15.27:0/1418838534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f01a2600 con 0x7f37f0105d90 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37ef400640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37f4e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f010c020 0x7f37f01a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:23:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.406+0000 7f37efe00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 0x7f37f019c690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37e000e9f0 tx=0x7f37e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.407+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e000cd80 con 0x7f37f0105d90 2024-09-21T01:23:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.407+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e00108e0 con 0x7f37f0105d90 2024-09-21T01:23:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.407+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01a28f0 con 0x7f37f0105d90 2024-09-21T01:23:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.407+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0018980 con 0x7f37f0105d90 2024-09-21T01:23:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.407+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01a9050 con 0x7f37f0105d90 2024-09-21T01:23:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.408+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f01061a0 con 0x7f37f0105d90 2024-09-21T01:23:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.411+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f37e0010430 con 0x7f37f0105d90 2024-09-21T01:23:03.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.414+0000 7f37ecc00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37d40b6710 0x7f37d40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:03.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.414+0000 7f37ef400640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37d40b6710 0x7f37d40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.414+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f37e00e84f0 con 0x7f37f0105d90 2024-09-21T01:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.414+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37e00ae500 con 0x7f37f0105d90 2024-09-21T01:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.415+0000 7f37ef400640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37d40b6710 0x7f37d40b8c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009230 tx=0x7f37dc000bc0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:03.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.676+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f002e640 con 0x7f37f0105d90 2024-09-21T01:23:03.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.677+0000 7f37ecc00640 1 -- 172.21.15.27:0/1418838534 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f37e00b28e0 con 0x7f37f0105d90 2024-09-21T01:23:03.678 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:23:03.681 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.681+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37d40b6710 msgr2=0x7f37d40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.681+0000 7f37f6e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37d40b6710 0x7f37d40b8c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009230 tx=0x7f37dc000bc0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.681+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 msgr2=0x7f37f019c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.681+0000 7f37f6e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 0x7f37f019c690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f37e000e9f0 tx=0x7f37e000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 shutdown_connections 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37d40b6710 0x7f37d40b8c20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37f010c020 0x7f37f01a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37f010b6d0 0x7f37f019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 --2- 172.21.15.27:0/1418838534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37f0105d90 0x7f37f019c690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 >> 172.21.15.27:0/1418838534 conn(0x7f37f0101860 msgr2=0x7f37f0103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 shutdown_connections 2024-09-21T01:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:03.682+0000 7f37f6e00640 1 -- 172.21.15.27:0/1418838534 wait complete. 2024-09-21T01:23:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:03 smithi027 ceph-mon[30375]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:23:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:03 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1418838534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:04.052 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:03 smithi089 ceph-mon[34486]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:23:04.052 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:03 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1418838534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:03 smithi098 ceph-mon[34372]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:23:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:03 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1418838534' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:04.593 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T01:23:04.593 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T01:23:05.028 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:23:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 -- 172.21.15.27:0/3079612816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 msgr2=0x7f3838110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 --2- 172.21.15.27:0/3079612816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 0x7f3838110140 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3830009ca0 tx=0x7f383002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 -- 172.21.15.27:0/3079612816 shutdown_connections 2024-09-21T01:23:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 --2- 172.21.15.27:0/3079612816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 0x7f3838110140 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 --2- 172.21.15.27:0/3079612816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3838105140 0x7f383810d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 --2- 172.21.15.27:0/3079612816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f38381047f0 0x7f3838104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 -- 172.21.15.27:0/3079612816 >> 172.21.15.27:0/3079612816 conn(0x7f38380fe330 msgr2=0x7f3838100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 -- 172.21.15.27:0/3079612816 shutdown_connections 2024-09-21T01:23:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.711+0000 7f3837200640 1 -- 172.21.15.27:0/3079612816 wait complete. 2024-09-21T01:23:05.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.712+0000 7f3837200640 1 Processor -- start 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 -- start start 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38381047f0 0x7f383819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 0x7f383819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 0x7f38381a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113920 con 0x7f383810dc50 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f38381137a0 con 0x7f3838105140 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3837200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3838113aa0 con 0x7f38381047f0 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 0x7f383819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3836800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 0x7f38381a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 0x7f383819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57502/0 (socket says 172.21.15.27:57502) 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3835e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38381047f0 0x7f383819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.713+0000 7f3835400640 1 -- 172.21.15.27:0/3909901331 learned_addr learned my addr 172.21.15.27:0/3909901331 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.714+0000 7f3835400640 1 -- 172.21.15.27:0/3909901331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38381047f0 msgr2=0x7f383819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.714+0000 7f3835400640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38381047f0 0x7f383819c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.714+0000 7f3835400640 1 -- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 msgr2=0x7f38381a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.714+0000 7f3835400640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 0x7f38381a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.714+0000 7f3835400640 1 -- 172.21.15.27:0/3909901331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38381a2420 con 0x7f3838105140 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.714+0000 7f3835400640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 0x7f383819ca50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f382400e9c0 tx=0x7f382400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:05.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.715+0000 7f382ea00640 1 -- 172.21.15.27:0/3909901331 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382400ccc0 con 0x7f3838105140 2024-09-21T01:23:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.715+0000 7f382ea00640 1 -- 172.21.15.27:0/3909901331 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38240054d0 con 0x7f3838105140 2024-09-21T01:23:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.715+0000 7f382ea00640 1 -- 172.21.15.27:0/3909901331 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824010430 con 0x7f3838105140 2024-09-21T01:23:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.715+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a2710 con 0x7f3838105140 2024-09-21T01:23:05.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.715+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381a8e70 con 0x7f3838105140 2024-09-21T01:23:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.718+0000 7f3843c14740 1 Processor -- start 2024-09-21T01:23:05.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.718+0000 7f3843c14740 1 -- start start 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3843c14740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3c1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3843c14740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3843c14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3843c14740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56112a336ca0 con 0x56112a019840 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3843c14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56112a3260d0 con 0x561129ff8b60 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3843c14740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56112a329ee0 con 0x561129ff0980 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3836800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3c1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3836800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3c1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35570/0 (socket says 172.21.15.27:35570) 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3836800640 1 -- 172.21.15.27:0/1678053821 learned_addr learned my addr 172.21.15.27:0/1678053821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:05.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f3835e00640 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.719+0000 7f382ea00640 1 -- 172.21.15.27:0/3909901331 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f3824005020 con 0x7f3838105140 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3836800640 1 -- 172.21.15.27:0/1678053821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 msgr2=0x56112a3c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3836800640 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3836800640 1 -- 172.21.15.27:0/1678053821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 msgr2=0x56112a3c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3836800640 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3836800640 1 -- 172.21.15.27:0/1678053821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112a2d8400 con 0x56112a019840 2024-09-21T01:23:05.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3836800640 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3c1510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3830009ca0 tx=0x7f3830009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54f12efdf8f108b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.720+0000 7f3823400640 1 -- 172.21.15.27:0/1678053821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830031cf0 con 0x56112a019840 2024-09-21T01:23:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.721+0000 7f3823400640 1 -- 172.21.15.27:0/1678053821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830036030 con 0x56112a019840 2024-09-21T01:23:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.721+0000 7f3823400640 1 -- 172.21.15.27:0/1678053821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830004280 con 0x56112a019840 2024-09-21T01:23:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f382ea00640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f381c0b6780 0x7f381c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 -- 172.21.15.27:0/1678053821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 msgr2=0x56112a3c1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3c1510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3830009ca0 tx=0x7f3830009430 comp rx=0 tx=0).stop 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f382ea00640 1 -- 172.21.15.27:0/3909901331 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f38240e7f70 con 0x7f3838105140 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 -- 172.21.15.27:0/1678053821 shutdown_connections 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 --2- 172.21.15.27:0/1678053821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3c1510 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 -- 172.21.15.27:0/1678053821 >> 172.21.15.27:0/1678053821 conn(0x56112a00d990 msgr2=0x56112a3bf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:05.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3835e00640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f381c0b6780 0x7f381c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.722+0000 7f3843c14740 1 -- 172.21.15.27:0/1678053821 shutdown_connections 2024-09-21T01:23:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.723+0000 7f3843c14740 1 -- 172.21.15.27:0/1678053821 wait complete. 2024-09-21T01:23:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.723+0000 7f3835e00640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f381c0b6780 0x7f381c0b8c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56112a3c0190 tx=0x7f3828005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.723+0000 7f3843c14740 1 -- start start 2024-09-21T01:23:05.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 Processor -- start 2024-09-21T01:23:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56112a1877a0 con 0x56112a019840 2024-09-21T01:23:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56112a2cba10 con 0x561129ff8b60 2024-09-21T01:23:05.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56112a3cfb10 con 0x561129ff0980 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.724+0000 7f3843c14740 10 client.0 _ll_register_callbacks cb 0x56112a2dbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:57514/0 (socket says 172.21.15.27:57514) 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 -- 172.21.15.27:0/1740526159 learned_addr learned my addr 172.21.15.27:0/1740526159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3836800640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835e00640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 -- 172.21.15.27:0/1740526159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 msgr2=0x56112a3ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 -- 172.21.15.27:0/1740526159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 msgr2=0x56112a3d8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3d8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 -- 172.21.15.27:0/1740526159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112a093b90 con 0x561129ff8b60 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3835400640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3cd130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f382400c6a0 tx=0x7f38240ad290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65269c75ff67260 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.725+0000 7f3817e00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:23:05.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3817e00640 1 -- 172.21.15.27:0/1740526159 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3824005310 con 0x561129ff8b60 2024-09-21T01:23:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3817e00640 1 -- 172.21.15.27:0/1740526159 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f38240aebb0 con 0x561129ff8b60 2024-09-21T01:23:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3830036030 con 0x561129ff8b60 2024-09-21T01:23:05.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3817e00640 1 -- 172.21.15.27:0/1740526159 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3824005680 con 0x561129ff8b60 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3843c14740 10 client.25652 fetch_fsmap learned FSMap version 29 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56112a3db870 con 0x561129ff8b60 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3817e00640 1 -- 172.21.15.27:0/1740526159 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830036030 con 0x561129ff8b60 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.726+0000 7f3817e00640 1 -- 172.21.15.27:0/1740526159 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f38240ac550 con 0x561129ff8b60 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3843c14740 10 client.25652 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3843c14740 10 client.25652 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3843c14740 20 client.25652 populate_metadata read hostname 'smithi027' 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3843c14740 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fd81c0 0x56112a3ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3843c14740 4 client.25652 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56112a338bf0 con 0x561129fd81c0 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.727+0000 7f3835e00640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fd81c0 0x56112a3ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.728+0000 7f3835e00640 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fd81c0 0x56112a3ded30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18e680c326afac9 server_cookie=bb36d2582b7ee44f in_seq=0 out_seq=0 2024-09-21T01:23:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.728+0000 7f3817e00640 10 client.25652 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.728+0000 7f3817e00640 1 -- 172.21.15.27:0/1740526159 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56112a338bf0 con 0x561129fd81c0 2024-09-21T01:23:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.728+0000 7f3817e00640 10 client.25652 handle_command_reply: tid=0 2024-09-21T01:23:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.728+0000 7f3843c14740 1 client.25652 shutdown 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 msgr2=0x56112a3cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3cd130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f382400c6a0 tx=0x7f38240ad290 comp rx=0 tx=0).stop 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 shutdown_connections 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56112a019840 0x56112a3d8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fd81c0 0x56112a3ded30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3cd130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 --2- 172.21.15.27:0/1740526159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3817e00640 0 client.25652 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 >> 172.21.15.27:0/1740526159 conn(0x56112a00d990 msgr2=0x56112a3c46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 shutdown_connections 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 1 -- 172.21.15.27:0/1740526159 wait complete. 2024-09-21T01:23:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.729+0000 7f3843c14740 20 client.25652 trim_cache size 0 max 16384 2024-09-21T01:23:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.748+0000 7f3843c14740 1 Processor -- start 2024-09-21T01:23:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 -- start start 2024-09-21T01:23:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff0980 0x56112a3c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 0x56112a3be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38381b2e10 con 0x561129ff0980 2024-09-21T01:23:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f38381b4950 con 0x561129ff8b60 2024-09-21T01:23:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3843c14740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38381b34e0 con 0x56112a019840 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3835e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff0980 0x56112a3c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.749+0000 7f3836800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 0x56112a3be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3835400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 0x56112a3be9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46968/0 (socket says 172.21.15.27:46968) 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 -- 172.21.15.27:0/3631357271 learned_addr learned my addr 172.21.15.27:0/3631357271 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 -- 172.21.15.27:0/3631357271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 msgr2=0x56112a3d8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3d8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 -- 172.21.15.27:0/3631357271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff0980 msgr2=0x56112a3c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff0980 0x56112a3c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3836800640 1 -- 172.21.15.27:0/3631357271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112a3bac00 con 0x56112a019840 2024-09-21T01:23:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.750+0000 7f3835e00640 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff0980 0x56112a3c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:23:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.751+0000 7f3836800640 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 0x56112a3be9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3830003d50 tx=0x7f3830003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68f259a5d9ef4e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.751+0000 7f3822a00640 1 -- 172.21.15.27:0/3631357271 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830056070 con 0x56112a019840 2024-09-21T01:23:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.751+0000 7f3822a00640 1 -- 172.21.15.27:0/3631357271 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830036040 con 0x56112a019840 2024-09-21T01:23:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.751+0000 7f3822a00640 1 -- 172.21.15.27:0/3631357271 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383004a040 con 0x56112a019840 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 -- 172.21.15.27:0/3631357271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 msgr2=0x56112a3be9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 0x56112a3be9b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3830003d50 tx=0x7f3830003d80 comp rx=0 tx=0).stop 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 -- 172.21.15.27:0/3631357271 shutdown_connections 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56112a019840 0x56112a3be9b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561129ff8b60 0x56112a3d8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 --2- 172.21.15.27:0/3631357271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff0980 0x56112a3c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 -- 172.21.15.27:0/3631357271 >> 172.21.15.27:0/3631357271 conn(0x56112a00d990 msgr2=0x56112a3d86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 -- 172.21.15.27:0/3631357271 shutdown_connections 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.752+0000 7f3843c14740 1 -- 172.21.15.27:0/3631357271 wait complete. 2024-09-21T01:23:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 -- start start 2024-09-21T01:23:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 Processor -- start 2024-09-21T01:23:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff8b60 0x56112a3d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56112a019840 0x56112a448690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38381b4280 con 0x561129ff8b60 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f38381b3bb0 con 0x56112a019840 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f38381b0df0 con 0x561129ff0980 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.753+0000 7f3843c14740 10 client.0 _ll_register_callbacks cb 0x7f38240e50c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff8b60 0x56112a3d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:46974/0 (socket says 172.21.15.27:46974) 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835e00640 1 -- 172.21.15.27:0/3585323674 learned_addr learned my addr 172.21.15.27:0/3585323674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3836800640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56112a019840 0x56112a448690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835400640 1 -- 172.21.15.27:0/3585323674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 msgr2=0x56112a3d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835400640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835400640 1 -- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56112a019840 msgr2=0x56112a448690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835400640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56112a019840 0x56112a448690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835400640 1 -- 172.21.15.27:0/3585323674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112a448bd0 con 0x561129ff8b60 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.754+0000 7f3835e00640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3835400640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff8b60 0x56112a3d3a10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38240ad810 tx=0x7f38240ad300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e024f9ddb693e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3817e00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3817e00640 1 -- 172.21.15.27:0/3585323674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38240aedb0 con 0x561129ff8b60 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3817e00640 1 -- 172.21.15.27:0/3585323674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f38240a91d0 con 0x561129ff8b60 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3830036040 con 0x561129ff8b60 2024-09-21T01:23:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3817e00640 1 -- 172.21.15.27:0/3585323674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38240b7090 con 0x561129ff8b60 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3843c14740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.755+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56112a449e30 con 0x561129ff8b60 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3817e00640 1 -- 172.21.15.27:0/3585323674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830036040 con 0x561129ff8b60 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3817e00640 1 -- 172.21.15.27:0/3585323674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f38240e5a40 con 0x561129ff8b60 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3843c14740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3843c14740 10 client.15618 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3843c14740 20 client.15618 populate_metadata read hostname 'smithi027' 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3843c14740 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fad570 0x56112a44c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3843c14740 4 client.15618 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T01:23:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x56112a3346a0 con 0x561129fad570 2024-09-21T01:23:05.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.756+0000 7f3835e00640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fad570 0x56112a44c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:05.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.757+0000 7f3835e00640 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fad570 0x56112a44c440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b315e7c679f257c4 server_cookie=a463b9598caf162f in_seq=0 out_seq=0 2024-09-21T01:23:05.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.757+0000 7f3817e00640 10 client.15618 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3817e00640 1 -- 172.21.15.27:0/3585323674 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56112a3346a0 con 0x561129fad570 2024-09-21T01:23:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3817e00640 10 client.15618 handle_command_reply: tid=0 2024-09-21T01:23:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 client.15618 shutdown 2024-09-21T01:23:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff8b60 msgr2=0x56112a3d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff8b60 0x56112a3d3a10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38240ad810 tx=0x7f38240ad300 comp rx=0 tx=0).stop 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 shutdown_connections 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x561129fad570 0x56112a44c440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56112a019840 0x56112a448690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561129ff8b60 0x56112a3d3a10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 --2- 172.21.15.27:0/3585323674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561129ff0980 0x56112a3d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 >> 172.21.15.27:0/3585323674 conn(0x56112a00d990 msgr2=0x56112a3d8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.770+0000 7f3817e00640 0 client.15618 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.771+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 shutdown_connections 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.771+0000 7f3843c14740 1 -- 172.21.15.27:0/3585323674 wait complete. 2024-09-21T01:23:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.771+0000 7f3843c14740 20 client.15618 trim_cache size 0 max 16384 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 2, 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "1dc72037": { 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 2, 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:23:05.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 134.5, 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:05.774 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f381c0b6780 msgr2=0x7f381c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f381c0b6780 0x7f381c0b8c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56112a3c0190 tx=0x7f3828005f90 comp rx=0 tx=0).stop 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 msgr2=0x7f383819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 0x7f383819ca50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f382400e9c0 tx=0x7f382400ee80 comp rx=0 tx=0).stop 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 shutdown_connections 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f381c0b6780 0x7f381c0b8c90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f383810dc50 0x7f38381a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3838105140 0x7f383819ca50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 --2- 172.21.15.27:0/3909901331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38381047f0 0x7f383819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 >> 172.21.15.27:0/3909901331 conn(0x7f38380fe330 msgr2=0x7f3838102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.772+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 shutdown_connections 2024-09-21T01:23:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:05.773+0000 7f3837200640 1 -- 172.21.15.27:0/3909901331 wait complete. 2024-09-21T01:23:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:05 smithi098 ceph-mon[34372]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:23:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:05 smithi027 ceph-mon[30375]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:23:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:05 smithi089 ceph-mon[34486]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:23:06.604 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1dc72037', quiesce took 4.3 seconds. Will release after: 10.2 2024-09-21T01:23:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:07 smithi098 ceph-mon[34372]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T01:23:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:07 smithi027 ceph-mon[30375]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T01:23:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:07 smithi089 ceph-mon[34486]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T01:23:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:09 smithi098 ceph-mon[34372]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T01:23:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:09 smithi027 ceph-mon[30375]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T01:23:10.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:09 smithi089 ceph-mon[34486]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T01:23:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:11 smithi098 ceph-mon[34372]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-21T01:23:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:11 smithi027 ceph-mon[30375]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-21T01:23:12.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:11 smithi089 ceph-mon[34486]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-21T01:23:14.128 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:13 smithi027 ceph-mon[30375]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-21T01:23:14.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:13 smithi098 ceph-mon[34372]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-21T01:23:14.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:13 smithi089 ceph-mon[34486]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-21T01:23:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:15 smithi098 ceph-mon[34372]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 23 op/s 2024-09-21T01:23:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:15 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:23:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:15 smithi027 ceph-mon[30375]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 23 op/s 2024-09-21T01:23:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:15 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:23:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:15 smithi089 ceph-mon[34486]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 23 op/s 2024-09-21T01:23:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:15 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:23:16.806 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:23:17.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:23:17.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:23:17.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:23:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:23:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:23:17.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:23:17.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:23:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:23:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:23:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:23:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.012+0000 7f2657200640 1 -- 172.21.15.27:0/336719337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f265010de30 msgr2=0x7f2650110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.012+0000 7f2657200640 1 --2- 172.21.15.27:0/336719337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f265010de30 0x7f2650110320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2648009ca0 tx=0x7f264802a8b0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 -- 172.21.15.27:0/336719337 shutdown_connections 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 --2- 172.21.15.27:0/336719337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f265010de30 0x7f2650110320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 --2- 172.21.15.27:0/336719337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2650105320 0x7f265010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 --2- 172.21.15.27:0/336719337 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 0x7f2650104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 -- 172.21.15.27:0/336719337 >> 172.21.15.27:0/336719337 conn(0x7f2650079740 msgr2=0x7f2650079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 -- 172.21.15.27:0/336719337 shutdown_connections 2024-09-21T01:23:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.013+0000 7f2657200640 1 -- 172.21.15.27:0/336719337 wait complete. 2024-09-21T01:23:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.014+0000 7f2657200640 1 Processor -- start 2024-09-21T01:23:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.014+0000 7f2657200640 1 -- start start 2024-09-21T01:23:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2657200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 0x7f265019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2657200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 0x7f265019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2657200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f265010de30 0x7f26501a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2657200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2650113b00 con 0x7f2650105320 2024-09-21T01:23:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2657200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2650113980 con 0x7f26501049d0 2024-09-21T01:23:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2657200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2650113c80 con 0x7f265010de30 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.015+0000 7f2647e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 0x7f265019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f264fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 0x7f265019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 0x7f265019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60874/0 (socket says 172.21.15.27:60874) 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 -- 172.21.15.27:0/1314200571 learned_addr learned my addr 172.21.15.27:0/1314200571 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2655200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f265010de30 0x7f26501a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 -- 172.21.15.27:0/1314200571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f265010de30 msgr2=0x7f26501a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f265010de30 0x7f26501a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 -- 172.21.15.27:0/1314200571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 msgr2=0x7f265019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 0x7f265019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f2647e00640 1 -- 172.21.15.27:0/1314200571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26501a26a0 con 0x7f2650105320 2024-09-21T01:23:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.016+0000 7f264fe00640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 0x7f265019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:23:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.017+0000 7f2647e00640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 0x7f265019ccd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f263800e9c0 tx=0x7f263800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.017+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263800ccc0 con 0x7f2650105320 2024-09-21T01:23:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.017+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26380054d0 con 0x7f2650105320 2024-09-21T01:23:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.017+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638010430 con 0x7f2650105320 2024-09-21T01:23:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.017+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26501a2990 con 0x7f2650105320 2024-09-21T01:23:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.017+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26501a90f0 con 0x7f2650105320 2024-09-21T01:23:18.021 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.019+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265006c240 con 0x7f2650105320 2024-09-21T01:23:18.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.024+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f26380046c0 con 0x7f2650105320 2024-09-21T01:23:18.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.026+0000 7f264d600640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f26340b6860 0x7f26340b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:18.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.026+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f26380e80d0 con 0x7f2650105320 2024-09-21T01:23:18.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.027+0000 7f264fe00640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f26340b6860 0x7f26340b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:18.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.028+0000 7f264fe00640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f26340b6860 0x7f26340b8d70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f265006c940 tx=0x7f2640005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:18.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.028+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2638056710 con 0x7f2650105320 2024-09-21T01:23:18.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:17 smithi098 ceph-mon[34372]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-21T01:23:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:17 smithi027 ceph-mon[30375]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-21T01:23:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:17 smithi089 ceph-mon[34486]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-21T01:23:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.273+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2650104de0 con 0x7f2650105320 2024-09-21T01:23:18.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.274+0000 7f264d600640 1 -- 172.21.15.27:0/1314200571 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f26380ae160 con 0x7f2650105320 2024-09-21T01:23:18.276 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:23:18.277 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:23:18.277 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:23:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.278+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f26340b6860 msgr2=0x7f26340b8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.278+0000 7f2657200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f26340b6860 0x7f26340b8d70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f265006c940 tx=0x7f2640005f90 comp rx=0 tx=0).stop 2024-09-21T01:23:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.278+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 msgr2=0x7f265019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.278+0000 7f2657200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 0x7f265019ccd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f263800e9c0 tx=0x7f263800ee80 comp rx=0 tx=0).stop 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.278+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 shutdown_connections 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.278+0000 7f2657200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f26340b6860 0x7f26340b8d70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.279+0000 7f2657200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f265010de30 0x7f26501a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.279+0000 7f2657200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2650105320 0x7f265019ccd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.279+0000 7f2657200640 1 --2- 172.21.15.27:0/1314200571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f26501049d0 0x7f265019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.279+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 >> 172.21.15.27:0/1314200571 conn(0x7f2650079740 msgr2=0x7f26501036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.279+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 shutdown_connections 2024-09-21T01:23:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:18.279+0000 7f2657200640 1 -- 172.21.15.27:0/1314200571 wait complete. 2024-09-21T01:23:19.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:18 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1314200571' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:19.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:18 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1314200571' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:19.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:18 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1314200571' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:19.273 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db --set-id 1dc72037 --release --await' 2024-09-21T01:23:19.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db --set-id 1dc72037 --release --await 2024-09-21T01:23:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:23:20.130 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:19 smithi027 ceph-mon[30375]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T01:23:20.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:19 smithi098 ceph-mon[34372]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T01:23:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:19 smithi089 ceph-mon[34486]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T01:23:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.630+0000 7f8e12e00640 1 -- 172.21.15.27:0/3867803028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c1047c0 msgr2=0x7f8e0c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.630+0000 7f8e12e00640 1 --2- 172.21.15.27:0/3867803028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c104bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009860 tx=0x7f8e0002a340 comp rx=0 tx=0).stop 2024-09-21T01:23:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 -- 172.21.15.27:0/3867803028 shutdown_connections 2024-09-21T01:23:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 --2- 172.21.15.27:0/3867803028 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c10dc20 0x7f8e0c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 --2- 172.21.15.27:0/3867803028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 0x7f8e0c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 --2- 172.21.15.27:0/3867803028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c104bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 -- 172.21.15.27:0/3867803028 >> 172.21.15.27:0/3867803028 conn(0x7f8e0c0fe2c0 msgr2=0x7f8e0c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:20.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 -- 172.21.15.27:0/3867803028 shutdown_connections 2024-09-21T01:23:20.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.631+0000 7f8e12e00640 1 -- 172.21.15.27:0/3867803028 wait complete. 2024-09-21T01:23:20.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.633+0000 7f8e12e00640 1 Processor -- start 2024-09-21T01:23:20.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.633+0000 7f8e12e00640 1 -- start start 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e12e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e12e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 0x7f8e0c1a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e12e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c10dc20 0x7f8e0c1ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e12e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c113880 con 0x7f8e0c105110 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e12e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c113700 con 0x7f8e0c1047c0 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e12e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c113a00 con 0x7f8e0c10dc20 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e11a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e11a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c1a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52660/0 (socket says 172.21.15.27:52660) 2024-09-21T01:23:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.634+0000 7f8e11a00640 1 -- 172.21.15.27:0/1867511634 learned_addr learned my addr 172.21.15.27:0/1867511634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e12400640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c10dc20 0x7f8e0c1ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11000640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 0x7f8e0c1a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11a00640 1 -- 172.21.15.27:0/1867511634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c10dc20 msgr2=0x7f8e0c1ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11a00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c10dc20 0x7f8e0c1ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11a00640 1 -- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 msgr2=0x7f8e0c1a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11a00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 0x7f8e0c1a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11a00640 1 -- 172.21.15.27:0/1867511634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c1acb20 con 0x7f8e0c1047c0 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.635+0000 7f8e11000640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 0x7f8e0c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.636+0000 7f8e11a00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c1a1560 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8e000303f0 tx=0x7f8e00032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.636+0000 7f8e0a000640 1 -- 172.21.15.27:0/1867511634 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00040020 con 0x7f8e0c1047c0 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.636+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c1accf0 con 0x7f8e0c1047c0 2024-09-21T01:23:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.636+0000 7f8e0a000640 1 -- 172.21.15.27:0/1867511634 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e00030dd0 con 0x7f8e0c1047c0 2024-09-21T01:23:20.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.636+0000 7f8e0a000640 1 -- 172.21.15.27:0/1867511634 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00037040 con 0x7f8e0c1047c0 2024-09-21T01:23:20.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.637+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1ad150 con 0x7f8e0c1047c0 2024-09-21T01:23:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.641+0000 7f8e1b0bb740 1 Processor -- start 2024-09-21T01:23:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.642+0000 7f8e1b0bb740 1 -- start start 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e0a000640 1 -- 172.21.15.27:0/1867511634 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f8e00043020 con 0x7f8e0c1047c0 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379ffa520 0x56037a3a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 0x56037a3a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56037a318290 con 0x560379ffa520 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56037a3076c0 con 0x560379fd1660 2024-09-21T01:23:20.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56037a30b4d0 con 0x560379fd9840 2024-09-21T01:23:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.643+0000 7f8e11000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.644+0000 7f8e11a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 0x56037a3a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.644+0000 7f8e11a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 0x56037a3a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52672/0 (socket says 172.21.15.27:52672) 2024-09-21T01:23:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.644+0000 7f8e11a00640 1 -- 172.21.15.27:0/814119678 learned_addr learned my addr 172.21.15.27:0/814119678 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e12400640 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379ffa520 0x56037a3a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e11a00640 1 -- 172.21.15.27:0/814119678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 msgr2=0x56037a3a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e11a00640 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e11a00640 1 -- 172.21.15.27:0/814119678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379ffa520 msgr2=0x56037a3a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e11a00640 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379ffa520 0x56037a3a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e11a00640 1 -- 172.21.15.27:0/814119678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037a294460 con 0x560379fd1660 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e11000640 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:23:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e12400640 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379ffa520 0x56037a3a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:23:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.645+0000 7f8e0a000640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8df40b6790 0x7f8df40b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.646+0000 7f8e0a000640 1 -- 172.21.15.27:0/1867511634 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8e00107920 con 0x7f8e0c1047c0 2024-09-21T01:23:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.646+0000 7f8e11a00640 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 0x56037a3a1880 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8e00104b50 tx=0x7f8e00091270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7cb0fa349b69d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.646+0000 7f8e12400640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8df40b6790 0x7f8df40b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.646+0000 7f8dfea00640 1 -- 172.21.15.27:0/814119678 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000cd000 con 0x560379fd1660 2024-09-21T01:23:20.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.646+0000 7f8dfea00640 1 -- 172.21.15.27:0/814119678 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e00098e50 con 0x560379fd1660 2024-09-21T01:23:20.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.647+0000 7f8dfea00640 1 -- 172.21.15.27:0/814119678 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000cc9a0 con 0x560379fd1660 2024-09-21T01:23:20.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.648+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/814119678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 msgr2=0x56037a3a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.648+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 0x56037a3a1880 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8e00104b50 tx=0x7f8e00091270 comp rx=0 tx=0).stop 2024-09-21T01:23:20.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.648+0000 7f8e12400640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8df40b6790 0x7f8df40b8ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005fd0 tx=0x7f8e04009500 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:20.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.649+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/814119678 shutdown_connections 2024-09-21T01:23:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.649+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.649+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379fd1660 0x56037a3a1880 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.649+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/814119678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379ffa520 0x56037a3a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.649+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/814119678 >> 172.21.15.27:0/814119678 conn(0x560379fee910 msgr2=0x56037a39f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.649+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/814119678 shutdown_connections 2024-09-21T01:23:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.650+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/814119678 wait complete. 2024-09-21T01:23:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.650+0000 7f8e1b0bb740 1 -- start start 2024-09-21T01:23:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.651+0000 7f8e1b0bb740 1 Processor -- start 2024-09-21T01:23:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.651+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56037a168bb0 con 0x560379fd1660 2024-09-21T01:23:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56037a2a26c0 con 0x560379ffa520 2024-09-21T01:23:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56037a318290 con 0x560379fd9840 2024-09-21T01:23:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e1b0bb740 10 client.0 _ll_register_callbacks cb 0x56037a2bd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:23:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e11a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e11a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60908/0 (socket says 172.21.15.27:60908) 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e11a00640 1 -- 172.21.15.27:0/143612482 learned_addr learned my addr 172.21.15.27:0/143612482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.652+0000 7f8e11000640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.653+0000 7f8e12400640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.653+0000 7f8e11000640 1 -- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 msgr2=0x56037a3b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.653+0000 7f8e11000640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.653+0000 7f8e11000640 1 -- 172.21.15.27:0/143612482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 msgr2=0x56037a3acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.653+0000 7f8e11000640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.653+0000 7f8e11000640 1 -- 172.21.15.27:0/143612482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037a065b00 con 0x560379fd9840 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.654+0000 7f8e11000640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3ad460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c101aa0 tx=0x7f8df8012b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7affdaa47b0ae75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.654+0000 7f8df3400640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:23:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.654+0000 7f8df3400640 1 -- 172.21.15.27:0/143612482 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8030060 con 0x560379fd9840 2024-09-21T01:23:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.654+0000 7f8df3400640 1 -- 172.21.15.27:0/143612482 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8df8004030 con 0x560379fd9840 2024-09-21T01:23:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.654+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e00098e50 con 0x560379fd9840 2024-09-21T01:23:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.655+0000 7f8df3400640 1 -- 172.21.15.27:0/143612482 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8df8037050 con 0x560379fd9840 2024-09-21T01:23:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.655+0000 7f8e1b0bb740 10 client.25657 fetch_fsmap learned FSMap version 29 2024-09-21T01:23:20.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.655+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56037a0dc960 con 0x560379fd9840 2024-09-21T01:23:20.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.656+0000 7f8df3400640 1 -- 172.21.15.27:0/143612482 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e00098e50 con 0x560379fd9840 2024-09-21T01:23:20.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.656+0000 7f8df3400640 1 -- 172.21.15.27:0/143612482 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8df80037b0 con 0x560379fd9840 2024-09-21T01:23:20.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.656+0000 7f8e1b0bb740 10 client.25657 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:23:20.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.656+0000 7f8e1b0bb740 10 client.25657 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:23:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.656+0000 7f8e1b0bb740 20 client.25657 populate_metadata read hostname 'smithi027' 2024-09-21T01:23:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.657+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379fb8ea0 0x56037a3bea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.657+0000 7f8e1b0bb740 4 client.25657 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:23:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.657+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56037a31a1e0 con 0x560379fb8ea0 2024-09-21T01:23:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.657+0000 7f8e11a00640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379fb8ea0 0x56037a3bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.658+0000 7f8e11a00640 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379fb8ea0 0x56037a3bea40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd2cb2650e3957ac server_cookie=eca04c486d62eb50 in_seq=0 out_seq=0 2024-09-21T01:23:20.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.658+0000 7f8df3400640 10 client.25657 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.659+0000 7f8df3400640 1 -- 172.21.15.27:0/143612482 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56037a31a1e0 con 0x560379fb8ea0 2024-09-21T01:23:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.659+0000 7f8df3400640 10 client.25657 handle_command_reply: tid=0 2024-09-21T01:23:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.659+0000 7f8e1b0bb740 1 client.25657 shutdown 2024-09-21T01:23:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 msgr2=0x56037a3ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3ad460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c101aa0 tx=0x7f8df8012b20 comp rx=0 tx=0).stop 2024-09-21T01:23:20.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 shutdown_connections 2024-09-21T01:23:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379fb8ea0 0x56037a3bea40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3ad460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/143612482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 >> 172.21.15.27:0/143612482 conn(0x560379fee910 msgr2=0x56037a3a4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:20.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8df3400640 0 client.25657 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.660+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 shutdown_connections 2024-09-21T01:23:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.661+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/143612482 wait complete. 2024-09-21T01:23:20.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.661+0000 7f8e1b0bb740 20 client.25657 trim_cache size 0 max 16384 2024-09-21T01:23:20.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.697+0000 7f8e1b0bb740 1 Processor -- start 2024-09-21T01:23:20.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 -- start start 2024-09-21T01:23:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a39f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1b0650 con 0x560379fd1660 2024-09-21T01:23:20.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1af1e0 con 0x560379ffa520 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.698+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1af8b0 con 0x560379fd9840 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:52684/0 (socket says 172.21.15.27:52684) 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 -- 172.21.15.27:0/2050442739 learned_addr learned my addr 172.21.15.27:0/2050442739 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e11a00640 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e11000640 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a39f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 -- 172.21.15.27:0/2050442739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 msgr2=0x56037a39f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a39f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 -- 172.21.15.27:0/2050442739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 msgr2=0x56037a3a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e12400640 1 -- 172.21.15.27:0/2050442739 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037a39e960 con 0x560379ffa520 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.699+0000 7f8e11a00640 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:23:20.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.700+0000 7f8e12400640 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e04003e90 tx=0x7f8e0401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db5fc5756bd7093 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.700+0000 7f8dfe000640 1 -- 172.21.15.27:0/2050442739 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004b20 con 0x560379ffa520 2024-09-21T01:23:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.700+0000 7f8dfe000640 1 -- 172.21.15.27:0/2050442739 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04004cc0 con 0x560379ffa520 2024-09-21T01:23:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.700+0000 7f8dfe000640 1 -- 172.21.15.27:0/2050442739 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04005440 con 0x560379ffa520 2024-09-21T01:23:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.701+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2050442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 msgr2=0x56037a3b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.701+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8980 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e04003e90 tx=0x7f8e0401b000 comp rx=0 tx=0).stop 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2050442739 shutdown_connections 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b8980 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a39f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2050442739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2050442739 >> 172.21.15.27:0/2050442739 conn(0x560379fee910 msgr2=0x56037a39eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2050442739 shutdown_connections 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.702+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2050442739 wait complete. 2024-09-21T01:23:20.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.703+0000 7f8e1b0bb740 1 -- start start 2024-09-21T01:23:20.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.703+0000 7f8e1b0bb740 1 Processor -- start 2024-09-21T01:23:20.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1a0ea0 con 0x560379fd1660 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1aff80 con 0x560379ffa520 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c1ad320 con 0x560379fd9840 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.704+0000 7f8e1b0bb740 10 client.0 _ll_register_callbacks cb 0x7f8df8009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.705+0000 7f8e11000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.705+0000 7f8e11000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:55532/0 (socket says 172.21.15.27:55532) 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.705+0000 7f8e11000640 1 -- 172.21.15.27:0/2384741065 learned_addr learned my addr 172.21.15.27:0/2384741065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.705+0000 7f8e12400640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.705+0000 7f8e11a00640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.706+0000 7f8e12400640 1 -- 172.21.15.27:0/2384741065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 msgr2=0x56037a3b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.706+0000 7f8e12400640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.706+0000 7f8e12400640 1 -- 172.21.15.27:0/2384741065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 msgr2=0x56037a3b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.706+0000 7f8e12400640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.706+0000 7f8e12400640 1 -- 172.21.15.27:0/2384741065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037a3b3490 con 0x560379ffa520 2024-09-21T01:23:20.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.706+0000 7f8e11a00640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:23:20.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.707+0000 7f8e12400640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b2f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e040049d0 tx=0x7f8e04004fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436c20fcd2240a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:20.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.708+0000 7f8df3400640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:23:20.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.708+0000 7f8df3400640 1 -- 172.21.15.27:0/2384741065 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400f7b0 con 0x560379ffa520 2024-09-21T01:23:20.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.708+0000 7f8df3400640 1 -- 172.21.15.27:0/2384741065 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8e0400f950 con 0x560379ffa520 2024-09-21T01:23:20.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.708+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e04004cc0 con 0x560379ffa520 2024-09-21T01:23:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.709+0000 7f8df3400640 1 -- 172.21.15.27:0/2384741065 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e0400fb50 con 0x560379ffa520 2024-09-21T01:23:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.709+0000 7f8e1b0bb740 10 client.25682 fetch_fsmap learned FSMap version 29 2024-09-21T01:23:20.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.709+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56037a42a060 con 0x560379ffa520 2024-09-21T01:23:20.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8df3400640 1 -- 172.21.15.27:0/2384741065 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04004cc0 con 0x560379ffa520 2024-09-21T01:23:20.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8df3400640 1 -- 172.21.15.27:0/2384741065 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8e0401f690 con 0x560379ffa520 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8e1b0bb740 10 client.25682 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8e1b0bb740 10 client.25682 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8e1b0bb740 20 client.25682 populate_metadata read hostname 'smithi027' 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379f8e250 0x56037a42c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8e1b0bb740 4 client.25682 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1dc72037", "release": true, "await": true}] 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.710+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1dc72037", "release": true, "await": true}) -- 0x56037a315c90 con 0x560379f8e250 2024-09-21T01:23:20.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.711+0000 7f8e11a00640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379f8e250 0x56037a42c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.712+0000 7f8e11a00640 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379f8e250 0x56037a42c690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8cf246c0bb7e2f server_cookie=85b2dc0b2c4f90dd in_seq=0 out_seq=0 2024-09-21T01:23:20.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.712+0000 7f8df3400640 10 client.25682 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:20.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.714+0000 7f8df3400640 1 -- 172.21.15.27:0/2384741065 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56037a315c90 con 0x560379f8e250 2024-09-21T01:23:20.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.714+0000 7f8df3400640 10 client.25682 handle_command_reply: tid=0 2024-09-21T01:23:20.720 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:23:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:23:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:23:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 4, 2024-09-21T01:23:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:23:20.720 INFO:teuthology.orchestra.run.smithi027.stdout: "1dc72037": { 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 4, 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 134.5, 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:23:20.721 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:23:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:20.722 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:20.722 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.714+0000 7f8e1b0bb740 1 client.25682 shutdown 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 msgr2=0x56037a3b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b2f50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e040049d0 tx=0x7f8e04004fd0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 shutdown_connections 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x560379f8e250 0x56037a42c690 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560379ffa520 0x56037a3b2f50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560379fd9840 0x56037a3b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 --2- 172.21.15.27:0/2384741065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560379fd1660 0x56037a3b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.715+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 >> 172.21.15.27:0/2384741065 conn(0x560379fee910 msgr2=0x56037a3a2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.716+0000 7f8df3400640 0 client.25682 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.716+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 shutdown_connections 2024-09-21T01:23:20.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.716+0000 7f8e1b0bb740 1 -- 172.21.15.27:0/2384741065 wait complete. 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.716+0000 7f8e1b0bb740 20 client.25682 trim_cache size 0 max 16384 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.719+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8df40b6790 msgr2=0x7f8df40b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.719+0000 7f8e12e00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8df40b6790 0x7f8df40b8ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005fd0 tx=0x7f8e04009500 comp rx=0 tx=0).stop 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.719+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 msgr2=0x7f8e0c1a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.719+0000 7f8e12e00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c1a1560 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8e000303f0 tx=0x7f8e00032da0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 shutdown_connections 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f8df40b6790 0x7f8df40b8ca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e0c10dc20 0x7f8e0c1ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e0c105110 0x7f8e0c1a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 --2- 172.21.15.27:0/1867511634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8e0c1047c0 0x7f8e0c1a1560 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 >> 172.21.15.27:0/1867511634 conn(0x7f8e0c0fe2c0 msgr2=0x7f8e0c102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 shutdown_connections 2024-09-21T01:23:20.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:20.720+0000 7f8e12e00640 1 -- 172.21.15.27:0/1867511634 wait complete. 2024-09-21T01:23:21.545 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1dc72037', total seconds elapsed: 19.3 2024-09-21T01:23:22.089 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:21 smithi027 ceph-mon[30375]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T01:23:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:21 smithi098 ceph-mon[34372]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T01:23:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:21 smithi089 ceph-mon[34486]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T01:23:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:23 smithi098 ceph-mon[34372]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:23 smithi027 ceph-mon[30375]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:24.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:23 smithi089 ceph-mon[34486]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:25 smithi098 ceph-mon[34372]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:23:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:25 smithi027 ceph-mon[30375]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:23:26.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:25 smithi089 ceph-mon[34486]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:23:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:27 smithi098 ceph-mon[34372]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T01:23:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:27 smithi027 ceph-mon[30375]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T01:23:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:27 smithi089 ceph-mon[34486]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-21T01:23:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:29 smithi098 ceph-mon[34372]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:29 smithi027 ceph-mon[30375]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:30.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:29 smithi089 ceph-mon[34486]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:31 smithi098 ceph-mon[34372]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:23:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:31 smithi027 ceph-mon[30375]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:23:32.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:31 smithi089 ceph-mon[34486]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:23:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:33 smithi098 ceph-mon[34372]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:33 smithi027 ceph-mon[30375]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:33 smithi089 ceph-mon[34486]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:35 smithi098 ceph-mon[34372]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:23:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:35 smithi027 ceph-mon[30375]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:23:36.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:35 smithi089 ceph-mon[34486]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:23:38.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:37 smithi098 ceph-mon[34372]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:23:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:37 smithi027 ceph-mon[30375]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:23:38.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:37 smithi089 ceph-mon[34486]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:23:40.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:39 smithi098 ceph-mon[34372]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:39 smithi027 ceph-mon[30375]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:40.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:39 smithi089 ceph-mon[34486]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:23:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:41 smithi098 ceph-mon[34372]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:23:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:41 smithi027 ceph-mon[30375]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:23:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:41 smithi089 ceph-mon[34486]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:23:44.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:43 smithi098 ceph-mon[34372]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:44.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:43 smithi098 ceph-mon[34372]: mgrmap e26: x(active, since 12m), standbys: z, y 2024-09-21T01:23:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:43 smithi027 ceph-mon[30375]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:43 smithi027 ceph-mon[30375]: mgrmap e26: x(active, since 12m), standbys: z, y 2024-09-21T01:23:44.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:43 smithi089 ceph-mon[34486]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:23:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:43 smithi089 ceph-mon[34486]: mgrmap e26: x(active, since 12m), standbys: z, y 2024-09-21T01:23:44.628 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 58% (7470/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 95% (12163/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T01:23:45.603 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T01:23:45.603 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.603 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T01:23:45.603 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-09-21T01:23:45.604 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T01:23:45.618 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T01:23:45.664 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T01:23:45.669 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T01:23:45.669 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T01:23:46.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:45 smithi098 ceph-mon[34372]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T01:23:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:45 smithi027 ceph-mon[30375]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T01:23:46.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:45 smithi089 ceph-mon[34486]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T01:23:46.417 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T01:23:46.519 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T01:23:46.588 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T01:23:46.592 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T01:23:46.592 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T01:23:46.659 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T01:23:46.667 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:23:46.667 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T01:23:46.746 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-21T01:23:46.748 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-21T01:23:46.748 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-21T01:23:46.838 INFO:tasks.workunit.client.0.smithi027.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-21T01:23:46.924 INFO:tasks.workunit.client.0.smithi027.stdout: Iozone: Performance Test of File I/O 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Version $Revision: 3.494 $ 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Compiled for 64 bit mode. 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Build: linux-AMD64 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T01:23:46.925 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Run began: Sat Sep 21 01:23:46 2024 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Include close in write timing 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Include fsync in write timing 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: File size set to 1048576 kB 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Record Size 16 kB 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Output is in kBytes/sec 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache line size set to 32 bytes. 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: File stride size set to 17 * record size. 2024-09-21T01:23:46.926 INFO:tasks.workunit.client.0.smithi027.stdout: Throughput test with 1 process 2024-09-21T01:23:46.927 INFO:tasks.workunit.client.0.smithi027.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-21T01:23:48.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:47 smithi098 ceph-mon[34372]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T01:23:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:47 smithi027 ceph-mon[30375]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T01:23:48.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:47 smithi089 ceph-mon[34486]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T01:23:48.474 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T01:23:48.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:23:49.160 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:23:49.938 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:49 smithi027 ceph-mon[30375]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T01:23:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:49 smithi098 ceph-mon[34372]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T01:23:50.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:49 smithi089 ceph-mon[34486]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-21T01:23:50.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 -- 172.21.15.27:0/4133302352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc105bc0 msgr2=0x7febdc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 --2- 172.21.15.27:0/4133302352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc105bc0 0x7febdc105fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7febcc009ca0 tx=0x7febcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 -- 172.21.15.27:0/4133302352 shutdown_connections 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 --2- 172.21.15.27:0/4133302352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febdc10be50 0x7febdc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 --2- 172.21.15.27:0/4133302352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febdc10b500 0x7febdc10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 --2- 172.21.15.27:0/4133302352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc105bc0 0x7febdc105fd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 -- 172.21.15.27:0/4133302352 >> 172.21.15.27:0/4133302352 conn(0x7febdc101630 msgr2=0x7febdc103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 -- 172.21.15.27:0/4133302352 shutdown_connections 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.150+0000 7febdb200640 1 -- 172.21.15.27:0/4133302352 wait complete. 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.152+0000 7febdb200640 1 Processor -- start 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.152+0000 7febdb200640 1 -- start start 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febdb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febdc105bc0 0x7febdc10faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febdb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febdc10b500 0x7febdc10ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febdb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 0x7febdc110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febdb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febdc115b60 con 0x7febdc10b500 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febdb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7febdc1159e0 con 0x7febdc10be50 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febdb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febdc115ce0 con 0x7febdc105bc0 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febda800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 0x7febdc110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febda800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 0x7febdc110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42128/0 (socket says 172.21.15.27:42128) 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.153+0000 7febda800640 1 -- 172.21.15.27:0/2840612961 learned_addr learned my addr 172.21.15.27:0/2840612961 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febda800640 1 -- 172.21.15.27:0/2840612961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febdc105bc0 msgr2=0x7febdc10faa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febda800640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febdc105bc0 0x7febdc10faa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febda800640 1 -- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febdc10b500 msgr2=0x7febdc10ffe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febda800640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febdc10b500 0x7febdc10ffe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febda800640 1 -- 172.21.15.27:0/2840612961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc1ac980 con 0x7febdc10be50 2024-09-21T01:23:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febda800640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 0x7febdc110520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febd400e9a0 tx=0x7febd400eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd400ccc0 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.154+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febd400ce60 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.155+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4010690 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.155+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc1acc70 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.155+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc1ad160 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.157+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc0052c0 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.163+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7febd40046c0 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.165+0000 7febd2a00640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7febc00b6700 0x7febc00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.166+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7febd40e7fe0 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.166+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7febd40e8440 con 0x7febdc10be50 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.166+0000 7febd9e00640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7febc00b6700 0x7febc00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.168+0000 7febd9e00640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7febc00b6700 0x7febc00b8c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7febcc02ade0 tx=0x7febcc005fd0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:50.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.507+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febbc0055b0 con 0x7febdc10be50 2024-09-21T01:23:50.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.508+0000 7febd2a00640 1 -- 172.21.15.27:0/2840612961 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7febd4004b70 con 0x7febdc10be50 2024-09-21T01:23:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:23:50.509 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:23:50.509 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7febc00b6700 msgr2=0x7febc00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7febc00b6700 0x7febc00b8c10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7febcc02ade0 tx=0x7febcc005fd0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 msgr2=0x7febdc110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 0x7febdc110520 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febd400e9a0 tx=0x7febd400eda0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 shutdown_connections 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7febc00b6700 0x7febc00b8c10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febdc10be50 0x7febdc110520 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febdc10b500 0x7febdc10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 --2- 172.21.15.27:0/2840612961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febdc105bc0 0x7febdc10faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:50.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 >> 172.21.15.27:0/2840612961 conn(0x7febdc101630 msgr2=0x7febdc103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:50.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 shutdown_connections 2024-09-21T01:23:50.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:50.510+0000 7febdb200640 1 -- 172.21.15.27:0/2840612961 wait complete. 2024-09-21T01:23:51.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:50 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2840612961' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:51.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:50 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2840612961' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:51.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:50 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2840612961' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:23:51.694 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T01:23:51.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T01:23:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:51 smithi027 ceph-mon[30375]: pgmap v443: 129 pgs: 129 active+clean; 157 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 17 op/s 2024-09-21T01:23:52.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:51 smithi089 ceph-mon[34486]: pgmap v443: 129 pgs: 129 active+clean; 157 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 17 op/s 2024-09-21T01:23:52.391 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:23:52.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:51 smithi098 ceph-mon[34372]: pgmap v443: 129 pgs: 129 active+clean; 157 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 17 op/s 2024-09-21T01:23:52.476 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:23:52.476 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 initial writers = 189024.95 kB/sec 2024-09-21T01:23:52.477 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 initial writers = 188999.46 kB/sec 2024-09-21T01:23:52.477 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 189024.95 kB/sec 2024-09-21T01:23:52.477 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 189024.95 kB/sec 2024-09-21T01:23:52.477 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 189024.95 kB/sec 2024-09-21T01:23:52.477 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:23:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:53 smithi027 ceph-mon[30375]: pgmap v444: 129 pgs: 129 active+clean; 449 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 37 MiB/s wr, 20 op/s 2024-09-21T01:23:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:53 smithi089 ceph-mon[34486]: pgmap v444: 129 pgs: 129 active+clean; 449 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 37 MiB/s wr, 20 op/s 2024-09-21T01:23:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:53 smithi098 ceph-mon[34372]: pgmap v444: 129 pgs: 129 active+clean; 449 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 37 MiB/s wr, 20 op/s 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 -- 172.21.15.27:0/2742532435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd558109e00 msgr2=0x7fd558110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 --2- 172.21.15.27:0/2742532435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd558109e00 0x7fd558110630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd550009900 tx=0x7fd55002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 -- 172.21.15.27:0/2742532435 shutdown_connections 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 --2- 172.21.15.27:0/2742532435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd558109e00 0x7fd558110630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 --2- 172.21.15.27:0/2742532435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5581094b0 0x7fd5581098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 --2- 172.21.15.27:0/2742532435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd558103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 -- 172.21.15.27:0/2742532435 >> 172.21.15.27:0/2742532435 conn(0x7fd5580ff5a0 msgr2=0x7fd5581019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:55.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 -- 172.21.15.27:0/2742532435 shutdown_connections 2024-09-21T01:23:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.057+0000 7fd55e000640 1 -- 172.21.15.27:0/2742532435 wait complete. 2024-09-21T01:23:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.058+0000 7fd55e000640 1 Processor -- start 2024-09-21T01:23:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.058+0000 7fd55e000640 1 -- start start 2024-09-21T01:23:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd55e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd55819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd55e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5581094b0 0x7fd55819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd55e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd558109e00 0x7fd5581a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd55e000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd558113ac0 con 0x7fd558109e00 2024-09-21T01:23:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd55e000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd558113940 con 0x7fd558103b70 2024-09-21T01:23:55.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd55e000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd558113c40 con 0x7fd5581094b0 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd557e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd558109e00 0x7fd5581a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd557400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd55819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd54ea00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5581094b0 0x7fd55819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd557e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd558109e00 0x7fd5581a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58178/0 (socket says 172.21.15.27:58178) 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd557e00640 1 -- 172.21.15.27:0/1909685844 learned_addr learned my addr 172.21.15.27:0/1909685844 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.059+0000 7fd557400640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd55819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42140/0 (socket says 172.21.15.27:42140) 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd557400640 1 -- 172.21.15.27:0/1909685844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5581094b0 msgr2=0x7fd55819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd557400640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5581094b0 0x7fd55819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd557400640 1 -- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd558109e00 msgr2=0x7fd5581a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd557400640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd558109e00 0x7fd5581a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd557400640 1 -- 172.21.15.27:0/1909685844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5581a25c0 con 0x7fd558103b70 2024-09-21T01:23:55.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd557400640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd55819c650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd54800ec80 tx=0x7fd54800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd554c00640 1 -- 172.21.15.27:0/1909685844 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548003b70 con 0x7fd558103b70 2024-09-21T01:23:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5581a28b0 con 0x7fd558103b70 2024-09-21T01:23:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd554c00640 1 -- 172.21.15.27:0/1909685844 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd548003d10 con 0x7fd558103b70 2024-09-21T01:23:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd554c00640 1 -- 172.21.15.27:0/1909685844 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548010910 con 0x7fd558103b70 2024-09-21T01:23:55.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.060+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5581a9010 con 0x7fd558103b70 2024-09-21T01:23:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.063+0000 7fd564c5c740 1 Processor -- start 2024-09-21T01:23:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 -- start start 2024-09-21T01:23:55.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd554c00640 1 -- 172.21.15.27:0/1909685844 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fd548004560 con 0x7fd558103b70 2024-09-21T01:23:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44dd840 0x55a5d4885350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d4885890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4885dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5d47faca0 con 0x55a5d44dd840 2024-09-21T01:23:55.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a5d47ea0d0 con 0x55a5d44bcb60 2024-09-21T01:23:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.064+0000 7fd564c5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5d47edee0 con 0x55a5d44b4980 2024-09-21T01:23:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd557e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44dd840 0x55a5d4885350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd557e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44dd840 0x55a5d4885350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58192/0 (socket says 172.21.15.27:58192) 2024-09-21T01:23:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd557e00640 1 -- 172.21.15.27:0/1682833657 learned_addr learned my addr 172.21.15.27:0/1682833657 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd54ea00640 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4885dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd557400640 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d4885890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd54ea00640 1 -- 172.21.15.27:0/1682833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 msgr2=0x55a5d4885890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd54ea00640 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d4885890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd54ea00640 1 -- 172.21.15.27:0/1682833657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44dd840 msgr2=0x55a5d4885350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd54ea00640 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44dd840 0x55a5d4885350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.065+0000 7fd54ea00640 1 -- 172.21.15.27:0/1682833657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d479c400 con 0x55a5d44bcb60 2024-09-21T01:23:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.066+0000 7fd54ea00640 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4885dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd55806c6e0 tx=0x7fd540011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6ba9e9ed15b3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.066+0000 7fd547e00640 1 -- 172.21.15.27:0/1682833657 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540004050 con 0x55a5d44bcb60 2024-09-21T01:23:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.066+0000 7fd547e00640 1 -- 172.21.15.27:0/1682833657 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5400054d0 con 0x55a5d44bcb60 2024-09-21T01:23:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.066+0000 7fd547e00640 1 -- 172.21.15.27:0/1682833657 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540004c80 con 0x55a5d44bcb60 2024-09-21T01:23:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.066+0000 7fd554c00640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd53c0b6680 0x7fd53c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd554c00640 1 -- 172.21.15.27:0/1909685844 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd5480e81b0 con 0x7fd558103b70 2024-09-21T01:23:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd557e00640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd53c0b6680 0x7fd53c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 -- 172.21.15.27:0/1682833657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 msgr2=0x55a5d4885dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4885dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd55806c6e0 tx=0x7fd540011810 comp rx=0 tx=0).stop 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 -- 172.21.15.27:0/1682833657 shutdown_connections 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4885dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d4885890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 --2- 172.21.15.27:0/1682833657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44dd840 0x55a5d4885350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 -- 172.21.15.27:0/1682833657 >> 172.21.15.27:0/1682833657 conn(0x55a5d44d1990 msgr2=0x55a5d4883240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 -- 172.21.15.27:0/1682833657 shutdown_connections 2024-09-21T01:23:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.067+0000 7fd564c5c740 1 -- 172.21.15.27:0/1682833657 wait complete. 2024-09-21T01:23:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.068+0000 7fd557e00640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd53c0b6680 0x7fd53c0b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd550005f70 tx=0x7fd550005a40 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.068+0000 7fd564c5c740 1 -- start start 2024-09-21T01:23:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.068+0000 7fd564c5c740 1 Processor -- start 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d48909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 0x55a5d4890f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 0x55a5d489ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5d464b7a0 con 0x55a5d44bcb60 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a5d478fa10 con 0x55a5d44dd840 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5d4893b30 con 0x55a5d44b4980 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd564c5c740 10 client.0 _ll_register_callbacks cb 0x55a5d479fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd54ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 0x55a5d4890f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd557e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 0x55a5d489ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd557e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 0x55a5d489ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:42148/0 (socket says 172.21.15.27:42148) 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd557400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d48909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd54ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 0x55a5d4890f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58202/0 (socket says 172.21.15.27:58202) 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.069+0000 7fd557e00640 1 -- 172.21.15.27:0/3118781821 learned_addr learned my addr 172.21.15.27:0/3118781821 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd54ea00640 1 -- 172.21.15.27:0/3118781821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 msgr2=0x55a5d48909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd54ea00640 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d48909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd54ea00640 1 -- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 msgr2=0x55a5d489ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd54ea00640 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 0x55a5d489ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd54ea00640 1 -- 172.21.15.27:0/3118781821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d4557b90 con 0x55a5d44bcb60 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd557e00640 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 0x55a5d489ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd54ea00640 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 0x55a5d4890f00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd540004880 tx=0x7fd540009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605cf0e46a23f8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.070+0000 7fd544c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:23:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd544c00640 1 -- 172.21.15.27:0/3118781821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540027030 con 0x55a5d44bcb60 2024-09-21T01:23:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5400054d0 con 0x55a5d44bcb60 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd544c00640 1 -- 172.21.15.27:0/3118781821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd5400035f0 con 0x55a5d44bcb60 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd544c00640 1 -- 172.21.15.27:0/3118781821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd540034050 con 0x55a5d44bcb60 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd564c5c740 10 client.15654 fetch_fsmap learned FSMap version 29 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5d489f800 con 0x55a5d44bcb60 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.071+0000 7fd544c00640 1 -- 172.21.15.27:0/3118781821 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5400054d0 con 0x55a5d44bcb60 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd544c00640 1 -- 172.21.15.27:0/3118781821 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd540004b80 con 0x55a5d44bcb60 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd564c5c740 10 client.15654 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd564c5c740 10 client.15654 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd564c5c740 20 client.15654 populate_metadata read hostname 'smithi027' 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd564c5c740 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d449c1c0 0x55a5d48a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd564c5c740 4 client.15654 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d47fcbf0 con 0x55a5d449c1c0 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.072+0000 7fd557400640 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d449c1c0 0x55a5d48a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.073+0000 7fd557400640 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d449c1c0 0x55a5d48a2bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1f6608724dacb5 server_cookie=bf8bcf65e98df727 in_seq=0 out_seq=0 2024-09-21T01:23:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.073+0000 7fd544c00640 10 client.15654 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd544c00640 1 -- 172.21.15.27:0/3118781821 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5d47fcbf0 con 0x55a5d449c1c0 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd544c00640 10 client.15654 handle_command_reply: tid=0 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 client.15654 shutdown 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 msgr2=0x55a5d4890f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 0x55a5d4890f00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd540004880 tx=0x7fd540009fa0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 shutdown_connections 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d449c1c0 0x55a5d48a2bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44dd840 0x55a5d489ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44bcb60 0x55a5d4890f00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 --2- 172.21.15.27:0/3118781821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44b4980 0x55a5d48909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 >> 172.21.15.27:0/3118781821 conn(0x55a5d44d1990 msgr2=0x55a5d48819a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd544c00640 0 client.15654 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 shutdown_connections 2024-09-21T01:23:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.074+0000 7fd564c5c740 1 -- 172.21.15.27:0/3118781821 wait complete. 2024-09-21T01:23:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.075+0000 7fd564c5c740 20 client.15654 trim_cache size 0 max 16384 2024-09-21T01:23:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 Processor -- start 2024-09-21T01:23:55.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 -- start start 2024-09-21T01:23:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d488dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d488e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b3010 con 0x55a5d44b4980 2024-09-21T01:23:55.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b4b50 con 0x55a5d44bcb60 2024-09-21T01:23:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd564c5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b36e0 con 0x55a5d44dd840 2024-09-21T01:23:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.093+0000 7fd557400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48929b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58204/0 (socket says 172.21.15.27:58204) 2024-09-21T01:23:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 -- 172.21.15.27:0/257680708 learned_addr learned my addr 172.21.15.27:0/257680708 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557e00640 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d488e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd54ea00640 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d488dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 -- 172.21.15.27:0/257680708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 msgr2=0x55a5d488e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d488e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 -- 172.21.15.27:0/257680708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 msgr2=0x55a5d488dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d488dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 -- 172.21.15.27:0/257680708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d487ec50 con 0x55a5d44b4980 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.094+0000 7fd557400640 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48929b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd5480e55f0 tx=0x7fd5480ea690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f805aaa2987fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.095+0000 7fd547400640 1 -- 172.21.15.27:0/257680708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480a9b30 con 0x55a5d44b4980 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.095+0000 7fd547400640 1 -- 172.21.15.27:0/257680708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5480ad7a0 con 0x55a5d44b4980 2024-09-21T01:23:55.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.095+0000 7fd547400640 1 -- 172.21.15.27:0/257680708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480a9400 con 0x55a5d44b4980 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 -- 172.21.15.27:0/257680708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 msgr2=0x55a5d48929b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48929b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd5480e55f0 tx=0x7fd5480ea690 comp rx=0 tx=0).stop 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 -- 172.21.15.27:0/257680708 shutdown_connections 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d488e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d488dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 --2- 172.21.15.27:0/257680708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48929b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 -- 172.21.15.27:0/257680708 >> 172.21.15.27:0/257680708 conn(0x55a5d44d1990 msgr2=0x55a5d4886d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 -- 172.21.15.27:0/257680708 shutdown_connections 2024-09-21T01:23:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 -- 172.21.15.27:0/257680708 wait complete. 2024-09-21T01:23:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 -- start start 2024-09-21T01:23:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.096+0000 7fd564c5c740 1 Processor -- start 2024-09-21T01:23:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4895a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d490c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b4480 con 0x55a5d44b4980 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b3db0 con 0x55a5d44bcb60 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b0ff0 con 0x55a5d44dd840 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd564c5c740 10 client.0 _ll_register_callbacks cb 0x7fd5480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd557400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd54ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4895a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd557400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58208/0 (socket says 172.21.15.27:58208) 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.097+0000 7fd557400640 1 -- 172.21.15.27:0/2271403011 learned_addr learned my addr 172.21.15.27:0/2271403011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557e00640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d490c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557400640 1 -- 172.21.15.27:0/2271403011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 msgr2=0x55a5d490c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557400640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d490c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557400640 1 -- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 msgr2=0x55a5d4895a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557400640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4895a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557400640 1 -- 172.21.15.27:0/2271403011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d490c9b0 con 0x55a5d44b4980 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd54ea00640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4895a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd557400640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48954f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd54800efc0 tx=0x7fd5480aa130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01510800d8bfcfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd544c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd544c00640 1 -- 172.21.15.27:0/2271403011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480d5060 con 0x55a5d44b4980 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd544c00640 1 -- 172.21.15.27:0/2271403011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd5480d5200 con 0x55a5d44b4980 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.098+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5480ad7a0 con 0x55a5d44b4980 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd544c00640 1 -- 172.21.15.27:0/2271403011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5480ad400 con 0x55a5d44b4980 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 10 client.15660 fetch_fsmap learned FSMap version 29 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5d490dc10 con 0x55a5d44b4980 2024-09-21T01:23:55.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd544c00640 1 -- 172.21.15.27:0/2271403011 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5480ad7a0 con 0x55a5d44b4980 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd544c00640 1 -- 172.21.15.27:0/2271403011 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd5480e5ce0 con 0x55a5d44b4980 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 10 client.15660 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 20 client.15660 populate_metadata read hostname 'smithi027' 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d4471570 0x55a5d49103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 4 client.15660 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T01:23:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.099+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x55a5d47f86a0 con 0x55a5d4471570 2024-09-21T01:23:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.100+0000 7fd54ea00640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d4471570 0x55a5d49103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:23:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.100+0000 7fd54ea00640 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d4471570 0x55a5d49103e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ce63b176ae546e server_cookie=a25fc0f0ef6b8ac3 in_seq=0 out_seq=0 2024-09-21T01:23:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.100+0000 7fd544c00640 10 client.15660 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd544c00640 1 -- 172.21.15.27:0/2271403011 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a5d47f86a0 con 0x55a5d4471570 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd544c00640 10 client.15660 handle_command_reply: tid=0 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 client.15660 shutdown 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 msgr2=0x55a5d48954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48954f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd54800efc0 tx=0x7fd5480aa130 comp rx=0 tx=0).stop 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 shutdown_connections 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5d44dd840 0x55a5d490c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55a5d4471570 0x55a5d49103e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a5d44bcb60 0x55a5d4895a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 --2- 172.21.15.27:0/2271403011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5d44b4980 0x55a5d48954f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 >> 172.21.15.27:0/2271403011 conn(0x55a5d44d1990 msgr2=0x55a5d4886d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd544c00640 0 client.15660 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 shutdown_connections 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 1 -- 172.21.15.27:0/2271403011 wait complete. 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.113+0000 7fd564c5c740 20 client.15660 trim_cache size 0 max 16384 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:23:55.115 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 6, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "f9f3a5f8": { 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 6, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.0, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:23:55.116 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:23:55.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd53c0b6680 msgr2=0x7fd53c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd53c0b6680 0x7fd53c0b8b90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd550005f70 tx=0x7fd550005a40 comp rx=0 tx=0).stop 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 msgr2=0x7fd55819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd55819c650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd54800ec80 tx=0x7fd54800c6a0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 shutdown_connections 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd53c0b6680 0x7fd53c0b8b90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd558109e00 0x7fd5581a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5581094b0 0x7fd55819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 --2- 172.21.15.27:0/1909685844 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd558103b70 0x7fd55819c650 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 >> 172.21.15.27:0/1909685844 conn(0x7fd5580ff5a0 msgr2=0x7fd5581069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.114+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 shutdown_connections 2024-09-21T01:23:55.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:23:55.115+0000 7fd55e000640 1 -- 172.21.15.27:0/1909685844 wait complete. 2024-09-21T01:23:55.996 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f9f3a5f8', quiesce took 7.5 seconds. Will release after: 9.5 2024-09-21T01:23:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:55 smithi027 ceph-mon[30375]: pgmap v445: 129 pgs: 129 active+clean; 797 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 66 MiB/s wr, 27 op/s 2024-09-21T01:23:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:55 smithi089 ceph-mon[34486]: pgmap v445: 129 pgs: 129 active+clean; 797 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 66 MiB/s wr, 27 op/s 2024-09-21T01:23:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:55 smithi098 ceph-mon[34372]: pgmap v445: 129 pgs: 129 active+clean; 797 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 66 MiB/s wr, 27 op/s 2024-09-21T01:23:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:57 smithi027 ceph-mon[30375]: pgmap v446: 129 pgs: 129 active+clean; 985 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 82 MiB/s wr, 34 op/s 2024-09-21T01:23:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:57 smithi089 ceph-mon[34486]: pgmap v446: 129 pgs: 129 active+clean; 985 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 82 MiB/s wr, 34 op/s 2024-09-21T01:23:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:57 smithi098 ceph-mon[34372]: pgmap v446: 129 pgs: 129 active+clean; 985 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 82 MiB/s wr, 34 op/s 2024-09-21T01:24:00.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:23:59 smithi027 ceph-mon[30375]: pgmap v447: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-09-21T01:24:00.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:23:59 smithi089 ceph-mon[34486]: pgmap v447: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-09-21T01:24:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:23:59 smithi098 ceph-mon[34372]: pgmap v447: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-09-21T01:24:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:01 smithi027 ceph-mon[30375]: pgmap v448: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-21T01:24:02.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:01 smithi089 ceph-mon[34486]: pgmap v448: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-21T01:24:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:01 smithi098 ceph-mon[34372]: pgmap v448: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-21T01:24:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:03 smithi027 ceph-mon[30375]: pgmap v449: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 72 MiB/s wr, 29 op/s 2024-09-21T01:24:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:03 smithi089 ceph-mon[34486]: pgmap v449: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 72 MiB/s wr, 29 op/s 2024-09-21T01:24:04.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:03 smithi098 ceph-mon[34372]: pgmap v449: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 72 MiB/s wr, 29 op/s 2024-09-21T01:24:05.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:24:05.893 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:24:06.241 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:05 smithi027 ceph-mon[30375]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 MiB/s wr, 36 op/s 2024-09-21T01:24:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:05 smithi089 ceph-mon[34486]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 MiB/s wr, 36 op/s 2024-09-21T01:24:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:05 smithi098 ceph-mon[34372]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 MiB/s wr, 36 op/s 2024-09-21T01:24:06.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 -- 172.21.15.27:0/1698970904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 msgr2=0x7fde601061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 --2- 172.21.15.27:0/1698970904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde601061a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde50009900 tx=0x7fde5002a410 comp rx=0 tx=0).stop 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 -- 172.21.15.27:0/1698970904 shutdown_connections 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 --2- 172.21.15.27:0/1698970904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010c020 0x7fde60112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 --2- 172.21.15.27:0/1698970904 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde6010b6d0 0x7fde6010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 --2- 172.21.15.27:0/1698970904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde601061a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.560+0000 7fde65e00640 1 -- 172.21.15.27:0/1698970904 >> 172.21.15.27:0/1698970904 conn(0x7fde60101860 msgr2=0x7fde60103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.561+0000 7fde65e00640 1 -- 172.21.15.27:0/1698970904 shutdown_connections 2024-09-21T01:24:06.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.561+0000 7fde65e00640 1 -- 172.21.15.27:0/1698970904 wait complete. 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 Processor -- start 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 -- start start 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde6019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 0x7fde6019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde6010c020 0x7fde601a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fde60115ce0 con 0x7fde60105d90 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fde60115b60 con 0x7fde6010c020 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde65e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fde60115e60 con 0x7fde6010b6d0 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.562+0000 7fde5ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde6019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 0x7fde6019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:06.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 0x7fde6019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:37750/0 (socket says 172.21.15.27:37750) 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 -- 172.21.15.27:0/4258088637 learned_addr learned my addr 172.21.15.27:0/4258088637 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde5ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde6019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38694/0 (socket says 172.21.15.27:38694) 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde5f400640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde6010c020 0x7fde601a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 -- 172.21.15.27:0/4258088637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde6010c020 msgr2=0x7fde601a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde6010c020 0x7fde601a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 -- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 msgr2=0x7fde6019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde6019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 -- 172.21.15.27:0/4258088637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde601a2640 con 0x7fde6010b6d0 2024-09-21T01:24:06.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.563+0000 7fde57e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 0x7fde6019cc70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fde4800e9f0 tx=0x7fde4800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:06.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.564+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4800ccc0 con 0x7fde6010b6d0 2024-09-21T01:24:06.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.564+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde480108e0 con 0x7fde6010b6d0 2024-09-21T01:24:06.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.564+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48018860 con 0x7fde6010b6d0 2024-09-21T01:24:06.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.564+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde601a2930 con 0x7fde6010b6d0 2024-09-21T01:24:06.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.564+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde601a9090 con 0x7fde6010b6d0 2024-09-21T01:24:06.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.565+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde601061a0 con 0x7fde6010b6d0 2024-09-21T01:24:06.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.568+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fde48010430 con 0x7fde6010b6d0 2024-09-21T01:24:06.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.570+0000 7fde64a00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde440b6680 0x7fde440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:06.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.570+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fde480e8240 con 0x7fde6010b6d0 2024-09-21T01:24:06.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.570+0000 7fde5ea00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde440b6680 0x7fde440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:06.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.571+0000 7fde5ea00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde440b6680 0x7fde440b8b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fde50031440 tx=0x7fde50009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:06.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.571+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde480b27a0 con 0x7fde6010b6d0 2024-09-21T01:24:06.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.808+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde6019d800 con 0x7fde6010b6d0 2024-09-21T01:24:06.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.809+0000 7fde64a00640 1 -- 172.21.15.27:0/4258088637 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fde6019d800 con 0x7fde6010b6d0 2024-09-21T01:24:06.810 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:24:06.811 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:24:06.811 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde440b6680 msgr2=0x7fde440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde440b6680 0x7fde440b8b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fde50031440 tx=0x7fde50009560 comp rx=0 tx=0).stop 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 msgr2=0x7fde6019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 0x7fde6019cc70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fde4800e9f0 tx=0x7fde4800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 shutdown_connections 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fde440b6680 0x7fde440b8b90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fde6010c020 0x7fde601a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde6010b6d0 0x7fde6019cc70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 --2- 172.21.15.27:0/4258088637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde60105d90 0x7fde6019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:06.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.812+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 >> 172.21.15.27:0/4258088637 conn(0x7fde60101860 msgr2=0x7fde6010ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.813+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 shutdown_connections 2024-09-21T01:24:06.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:06.813+0000 7fde65e00640 1 -- 172.21.15.27:0/4258088637 wait complete. 2024-09-21T01:24:07.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:06 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4258088637' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:24:07.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:06 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4258088637' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:24:07.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:06 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4258088637' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:24:07.697 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db --set-id f9f3a5f8 --release --await' 2024-09-21T01:24:07.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db --set-id f9f3a5f8 --release --await 2024-09-21T01:24:08.088 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:24:08.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:07 smithi027 ceph-mon[30375]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 32 op/s 2024-09-21T01:24:08.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:07 smithi089 ceph-mon[34486]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 32 op/s 2024-09-21T01:24:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:07 smithi098 ceph-mon[34372]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 32 op/s 2024-09-21T01:24:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 -- 172.21.15.27:0/435044848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010b6c0 msgr2=0x7fe9c010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/435044848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010b6c0 0x7fe9c010bad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009900 tx=0x7fe9a802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 -- 172.21.15.27:0/435044848 shutdown_connections 2024-09-21T01:24:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/435044848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c010c010 0x7fe9c0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/435044848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010b6c0 0x7fe9c010bad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/435044848 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 0x7fe9c0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 -- 172.21.15.27:0/435044848 >> 172.21.15.27:0/435044848 conn(0x7fe9c0101810 msgr2=0x7fe9c0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 -- 172.21.15.27:0/435044848 shutdown_connections 2024-09-21T01:24:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.776+0000 7fe9c6a00640 1 -- 172.21.15.27:0/435044848 wait complete. 2024-09-21T01:24:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.777+0000 7fe9c6a00640 1 Processor -- start 2024-09-21T01:24:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.777+0000 7fe9c6a00640 1 -- start start 2024-09-21T01:24:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 0x7fe9c019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c010b6c0 0x7fe9c019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 0x7fe9c01a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c6a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0115cd0 con 0x7fe9c010c010 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c6a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0115b50 con 0x7fe9c0105d80 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c6a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0115e50 con 0x7fe9c010b6c0 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 0x7fe9c01a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 0x7fe9c01a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38718/0 (socket says 172.21.15.27:38718) 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9c4a00640 1 -- 172.21.15.27:0/3836960439 learned_addr learned my addr 172.21.15.27:0/3836960439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 0x7fe9c019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.778+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c010b6c0 0x7fe9c019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3836960439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c010b6c0 msgr2=0x7fe9c019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c010b6c0 0x7fe9c019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 msgr2=0x7fe9c01a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 0x7fe9c01a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3836960439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c01a2610 con 0x7fe9c0105d80 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9c4a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 0x7fe9c01a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 0x7fe9c019c6a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000e9f0 tx=0x7fe9b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bd600640 1 -- 172.21.15.27:0/3836960439 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000cd80 con 0x7fe9c0105d80 2024-09-21T01:24:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c01a2900 con 0x7fe9c0105d80 2024-09-21T01:24:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bd600640 1 -- 172.21.15.27:0/3836960439 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b00035f0 con 0x7fe9c0105d80 2024-09-21T01:24:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9bd600640 1 -- 172.21.15.27:0/3836960439 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0010690 con 0x7fe9c0105d80 2024-09-21T01:24:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.779+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c01a9060 con 0x7fe9c0105d80 2024-09-21T01:24:08.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.783+0000 7fe9cd5fb740 1 Processor -- start 2024-09-21T01:24:08.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 -- start start 2024-09-21T01:24:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb021a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb021a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe52840 0x559eb021ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559eb0191290 con 0x559eafe73520 2024-09-21T01:24:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559eb01806c0 con 0x559eafe4a660 2024-09-21T01:24:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.784+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559eb01844d0 con 0x559eafe52840 2024-09-21T01:24:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb021a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb021a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38728/0 (socket says 172.21.15.27:38728) 2024-09-21T01:24:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 -- 172.21.15.27:0/751571874 learned_addr learned my addr 172.21.15.27:0/751571874 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9b7400640 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe52840 0x559eb021ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 -- 172.21.15.27:0/751571874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe52840 msgr2=0x559eb021ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe52840 0x559eb021ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 -- 172.21.15.27:0/751571874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 msgr2=0x559eb021a700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb021a700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 -- 172.21.15.27:0/751571874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb010d460 con 0x559eafe73520 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9bd600640 1 -- 172.21.15.27:0/3836960439 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fe9b00037a0 con 0x7fe9c0105d80 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.785+0000 7fe9c4a00640 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb021a1c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80117e0 tx=0x7fe9b8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47347010ea82678d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.786+0000 7fe9b5200640 1 -- 172.21.15.27:0/751571874 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800fb40 con 0x559eafe73520 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.786+0000 7fe9b5200640 1 -- 172.21.15.27:0/751571874 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b800fce0 con 0x559eafe73520 2024-09-21T01:24:08.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.786+0000 7fe9b5200640 1 -- 172.21.15.27:0/751571874 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8010630 con 0x559eafe73520 2024-09-21T01:24:08.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/751571874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 msgr2=0x559eb021a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb021a1c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80117e0 tx=0x7fe9b8011ca0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/751571874 shutdown_connections 2024-09-21T01:24:08.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe52840 0x559eb021ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb021a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/751571874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb021a1c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/751571874 >> 172.21.15.27:0/751571874 conn(0x559eafe67910 msgr2=0x559eb02180b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:08.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/751571874 shutdown_connections 2024-09-21T01:24:08.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9bd600640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe9a40b6590 0x7fe9a40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.787+0000 7fe9bd600640 1 -- 172.21.15.27:0/3836960439 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe9b00e7df0 con 0x7fe9c0105d80 2024-09-21T01:24:08.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.788+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/751571874 wait complete. 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.788+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe9a40b6590 0x7fe9a40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.788+0000 7fe9cd5fb740 1 -- start start 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 Processor -- start 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe9a40b6590 0x7fe9a40b8aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009230 tx=0x7fe9a8031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb022ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb022f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb02342b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559eaffe1bb0 con 0x559eafe52840 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559eb011b6c0 con 0x559eafe4a660 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559eb02289a0 con 0x559eafe73520 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.789+0000 7fe9cd5fb740 10 client.0 _ll_register_callbacks cb 0x559eb0136490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb022ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb022ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39092/0 (socket says 172.21.15.27:39092) 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3210272654 learned_addr learned my addr 172.21.15.27:0/3210272654 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:08.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9c4a00640 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb02342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3210272654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 msgr2=0x559eb02342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb02342b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3210272654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 msgr2=0x559eb022f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb022f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 -- 172.21.15.27:0/3210272654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eafedeb00 con 0x559eafe4a660 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb022ecb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9b001f000 tx=0x7fe9b0009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4469444a2460e829 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9ad600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9ad600640 1 -- 172.21.15.27:0/3210272654 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00b2350 con 0x559eafe4a660 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b800fce0 con 0x559eafe4a660 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.790+0000 7fe9ad600640 1 -- 172.21.15.27:0/3210272654 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe9b00a91c0 con 0x559eafe4a660 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9ad600640 1 -- 172.21.15.27:0/3210272654 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9b00b7290 con 0x559eafe4a660 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 10 client.25721 fetch_fsmap learned FSMap version 29 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559eaff55960 con 0x559eafe4a660 2024-09-21T01:24:08.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9ad600640 1 -- 172.21.15.27:0/3210272654 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b800fce0 con 0x559eafe4a660 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9ad600640 1 -- 172.21.15.27:0/3210272654 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe9b00b34e0 con 0x559eafe4a660 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 10 client.25721 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 10 client.25721 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 20 client.25721 populate_metadata read hostname 'smithi027' 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe31ea0 0x559eb02405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 4 client.25721 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.791+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eb01931e0 con 0x559eafe31ea0 2024-09-21T01:24:08.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.792+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe31ea0 0x559eb02405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.792+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe31ea0 0x559eb02405a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbf5a157a69cc53 server_cookie=253c77483484fc4e in_seq=0 out_seq=0 2024-09-21T01:24:08.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.792+0000 7fe9ad600640 10 client.25721 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.793+0000 7fe9ad600640 1 -- 172.21.15.27:0/3210272654 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559eb01931e0 con 0x559eafe31ea0 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.793+0000 7fe9ad600640 10 client.25721 handle_command_reply: tid=0 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.793+0000 7fe9cd5fb740 1 client.25721 shutdown 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 msgr2=0x559eb022ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb022ecb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe9b001f000 tx=0x7fe9b0009590 comp rx=0 tx=0).stop 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 shutdown_connections 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb02342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb022f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe31ea0 0x559eb02405a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3210272654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb022ecb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 >> 172.21.15.27:0/3210272654 conn(0x559eafe67910 msgr2=0x559eb021d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9ad600640 0 client.25721 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 shutdown_connections 2024-09-21T01:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3210272654 wait complete. 2024-09-21T01:24:08.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.794+0000 7fe9cd5fb740 20 client.25721 trim_cache size 0 max 16384 2024-09-21T01:24:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 Processor -- start 2024-09-21T01:24:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 -- start start 2024-09-21T01:24:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb0218270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb023b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb0230990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01aebb0 con 0x559eafe52840 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b06f0 con 0x559eafe4a660 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.812+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01af280 con 0x559eafe73520 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb0218270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9b7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb023b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb0218270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:39100/0 (socket says 172.21.15.27:39100) 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 -- 172.21.15.27:0/1654286474 learned_addr learned my addr 172.21.15.27:0/1654286474 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:08.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9c4a00640 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb0230990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 -- 172.21.15.27:0/1654286474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 msgr2=0x559eb0230990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb0230990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 -- 172.21.15.27:0/1654286474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 msgr2=0x559eb023b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb023b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.813+0000 7fe9bfe00640 1 -- 172.21.15.27:0/1654286474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb021ca50 con 0x559eafe4a660 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.814+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb0218270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003e40 tx=0x7fe9b00a9110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f28c429be2c4ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.814+0000 7fe9afe00640 1 -- 172.21.15.27:0/1654286474 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00ad1c0 con 0x559eafe4a660 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.814+0000 7fe9afe00640 1 -- 172.21.15.27:0/1654286474 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b00b2350 con 0x559eafe4a660 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.814+0000 7fe9afe00640 1 -- 172.21.15.27:0/1654286474 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00a9ae0 con 0x559eafe4a660 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/1654286474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 msgr2=0x559eb0218270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb0218270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0003e40 tx=0x7fe9b00a9110 comp rx=0 tx=0).stop 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/1654286474 shutdown_connections 2024-09-21T01:24:08.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe73520 0x559eb0230990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe52840 0x559eb023b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/1654286474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe4a660 0x559eb0218270 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/1654286474 >> 172.21.15.27:0/1654286474 conn(0x559eafe67910 msgr2=0x559eb0216650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/1654286474 shutdown_connections 2024-09-21T01:24:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/1654286474 wait complete. 2024-09-21T01:24:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.815+0000 7fe9cd5fb740 1 -- start start 2024-09-21T01:24:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 Processor -- start 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe4a660 0x559eb023aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe52840 0x559eb0235490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb02359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b0020 con 0x559eafe73520 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01af950 con 0x559eafe52840 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01acb90 con 0x559eafe4a660 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9cd5fb740 10 client.0 _ll_register_callbacks cb 0x7fe9b00a9ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9c4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb02359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9b7400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe52840 0x559eb0235490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9c4a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb02359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38750/0 (socket says 172.21.15.27:38750) 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9c4a00640 1 -- 172.21.15.27:0/3604432805 learned_addr learned my addr 172.21.15.27:0/3604432805 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.816+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe4a660 0x559eb023aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9b7400640 1 -- 172.21.15.27:0/3604432805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe4a660 msgr2=0x559eb023aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe4a660 0x559eb023aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9b7400640 1 -- 172.21.15.27:0/3604432805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 msgr2=0x559eb02359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb02359d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9b7400640 1 -- 172.21.15.27:0/3604432805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb021e2d0 con 0x559eafe52840 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9b7400640 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe52840 0x559eb0235490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9a802a910 tx=0x7fe9a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d5d320dcdc534b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9ad600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.817+0000 7fe9ad600640 1 -- 172.21.15.27:0/3604432805 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8035040 con 0x559eafe52840 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b00b2350 con 0x559eafe52840 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9ad600640 1 -- 172.21.15.27:0/3604432805 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe9a8004240 con 0x559eafe52840 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9ad600640 1 -- 172.21.15.27:0/3604432805 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe9a8061050 con 0x559eafe52840 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9cd5fb740 10 client.25727 fetch_fsmap learned FSMap version 29 2024-09-21T01:24:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559eb023d140 con 0x559eafe52840 2024-09-21T01:24:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9ad600640 1 -- 172.21.15.27:0/3604432805 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b00b2350 con 0x559eafe52840 2024-09-21T01:24:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.818+0000 7fe9ad600640 1 -- 172.21.15.27:0/3604432805 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe9a8004a80 con 0x559eafe52840 2024-09-21T01:24:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9cd5fb740 10 client.25727 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:24:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9cd5fb740 10 client.25727 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:24:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9cd5fb740 20 client.25727 populate_metadata read hostname 'smithi027' 2024-09-21T01:24:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe07250 0x559eb02ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9cd5fb740 4 client.25727 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f9f3a5f8", "release": true, "await": true}] 2024-09-21T01:24:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f9f3a5f8", "release": true, "await": true}) -- 0x559eb018ec90 con 0x559eafe07250 2024-09-21T01:24:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.819+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe07250 0x559eb02ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.820+0000 7fe9bfe00640 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe07250 0x559eb02ae300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34e2aa6d46e8e63 server_cookie=71b7f10f7f8859df in_seq=0 out_seq=0 2024-09-21T01:24:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.820+0000 7fe9ad600640 10 client.25727 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9ad600640 1 -- 172.21.15.27:0/3604432805 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559eb018ec90 con 0x559eafe07250 2024-09-21T01:24:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9ad600640 10 client.25727 handle_command_reply: tid=0 2024-09-21T01:24:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 client.25727 shutdown 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe52840 msgr2=0x559eb0235490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe52840 0x559eb0235490 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9a802a910 tx=0x7fe9a8009260 comp rx=0 tx=0).stop 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 shutdown_connections 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559eafe73520 0x559eb02359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x559eafe07250 0x559eb02ae300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559eafe52840 0x559eb0235490 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 --2- 172.21.15.27:0/3604432805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559eafe4a660 0x559eb023aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9ad600640 0 client.25727 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 >> 172.21.15.27:0/3604432805 conn(0x559eafe67910 msgr2=0x559eb02306f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 shutdown_connections 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.823+0000 7fe9cd5fb740 1 -- 172.21.15.27:0/3604432805 wait complete. 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9cd5fb740 20 client.25727 trim_cache size 0 max 16384 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 8, 2024-09-21T01:24:08.825 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "f9f3a5f8": { 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 8, 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 137.0, 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:24:08.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:08.827 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe9a40b6590 msgr2=0x7fe9a40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe9a40b6590 0x7fe9a40b8aa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009230 tx=0x7fe9a8031400 comp rx=0 tx=0).stop 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 msgr2=0x7fe9c019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 0x7fe9c019c6a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000e9f0 tx=0x7fe9b000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 shutdown_connections 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe9a40b6590 0x7fe9a40b8aa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9c010c010 0x7fe9c01a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c010b6c0 0x7fe9c019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 --2- 172.21.15.27:0/3836960439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9c0105d80 0x7fe9c019c6a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.824+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 >> 172.21.15.27:0/3836960439 conn(0x7fe9c0101810 msgr2=0x7fe9c0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.825+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 shutdown_connections 2024-09-21T01:24:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:08.825+0000 7fe9c6a00640 1 -- 172.21.15.27:0/3836960439 wait complete. 2024-09-21T01:24:09.807 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f9f3a5f8', total seconds elapsed: 21.3 2024-09-21T01:24:10.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:09 smithi027 ceph-mon[30375]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 25 op/s 2024-09-21T01:24:10.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:09 smithi089 ceph-mon[34486]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 25 op/s 2024-09-21T01:24:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:09 smithi098 ceph-mon[34372]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 25 op/s 2024-09-21T01:24:12.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:11 smithi089 ceph-mon[34486]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 29 op/s 2024-09-21T01:24:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:11 smithi098 ceph-mon[34372]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 29 op/s 2024-09-21T01:24:12.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:11 smithi027 ceph-mon[30375]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.4 MiB/s wr, 29 op/s 2024-09-21T01:24:13.045 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:13.046 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 rewriters = 63376.20 kB/sec 2024-09-21T01:24:13.046 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 rewriters = 63373.45 kB/sec 2024-09-21T01:24:13.046 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 63376.20 kB/sec 2024-09-21T01:24:13.046 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 63376.20 kB/sec 2024-09-21T01:24:13.046 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 63376.20 kB/sec 2024-09-21T01:24:13.046 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:13 smithi089 ceph-mon[34486]: pgmap v454: 129 pgs: 129 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-09-21T01:24:14.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:13 smithi098 ceph-mon[34372]: pgmap v454: 129 pgs: 129 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-09-21T01:24:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:13 smithi027 ceph-mon[30375]: pgmap v454: 129 pgs: 129 active+clean; 1.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 MiB/s wr, 30 op/s 2024-09-21T01:24:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:15 smithi089 ceph-mon[34486]: pgmap v455: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 56 MiB/s wr, 36 op/s 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 readers = 3432767.00 kB/sec 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 readers = 3425398.44 kB/sec 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 3432767.00 kB/sec 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 3432767.00 kB/sec 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 3432767.00 kB/sec 2024-09-21T01:24:16.379 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:15 smithi098 ceph-mon[34372]: pgmap v455: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 56 MiB/s wr, 36 op/s 2024-09-21T01:24:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:15 smithi027 ceph-mon[30375]: pgmap v455: 129 pgs: 129 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 56 MiB/s wr, 36 op/s 2024-09-21T01:24:17.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:16 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:24:17.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:16 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:24:17.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:16 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:24:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:17 smithi089 ceph-mon[34486]: pgmap v456: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 78 MiB/s wr, 31 op/s 2024-09-21T01:24:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:24:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:24:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:17 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:24:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:17 smithi098 ceph-mon[34372]: pgmap v456: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 78 MiB/s wr, 31 op/s 2024-09-21T01:24:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:24:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:24:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:17 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:24:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:17 smithi027 ceph-mon[30375]: pgmap v456: 129 pgs: 129 active+clean; 1.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 78 MiB/s wr, 31 op/s 2024-09-21T01:24:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:24:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:24:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:17 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:24:18.879 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:18.879 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 re-readers = 2583501.50 kB/sec 2024-09-21T01:24:18.879 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 re-readers = 2579065.58 kB/sec 2024-09-21T01:24:18.880 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 2583501.50 kB/sec 2024-09-21T01:24:18.880 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 2583501.50 kB/sec 2024-09-21T01:24:18.880 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 2583501.50 kB/sec 2024-09-21T01:24:18.880 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:20 smithi098 ceph-mon[34372]: pgmap v457: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-09-21T01:24:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:20 smithi027 ceph-mon[30375]: pgmap v457: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-09-21T01:24:20.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:20 smithi089 ceph-mon[34486]: pgmap v457: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 MiB/s wr, 30 op/s 2024-09-21T01:24:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:22 smithi098 ceph-mon[34372]: pgmap v458: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-21T01:24:22.497 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:22 smithi027 ceph-mon[30375]: pgmap v458: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-21T01:24:22.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:22 smithi089 ceph-mon[34486]: pgmap v458: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-21T01:24:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:24 smithi098 ceph-mon[34372]: pgmap v459: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 81 MiB/s wr, 29 op/s 2024-09-21T01:24:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:24 smithi027 ceph-mon[30375]: pgmap v459: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 81 MiB/s wr, 29 op/s 2024-09-21T01:24:24.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:24 smithi089 ceph-mon[34486]: pgmap v459: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 81 MiB/s wr, 29 op/s 2024-09-21T01:24:25.396 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:25.396 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:25.397 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:25.397 INFO:tasks.workunit.client.0.smithi027.stdout:iozone test complete. 2024-09-21T01:24:25.400 INFO:tasks.workunit.client.0.smithi027.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-21T01:24:25.481 INFO:tasks.workunit.client.0.smithi027.stdout: Iozone: Performance Test of File I/O 2024-09-21T01:24:25.481 INFO:tasks.workunit.client.0.smithi027.stdout: Version $Revision: 3.494 $ 2024-09-21T01:24:25.481 INFO:tasks.workunit.client.0.smithi027.stdout: Compiled for 64 bit mode. 2024-09-21T01:24:25.481 INFO:tasks.workunit.client.0.smithi027.stdout: Build: linux-AMD64 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Run began: Sat Sep 21 01:24:25 2024 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:25.482 INFO:tasks.workunit.client.0.smithi027.stdout: Include close in write timing 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Include fsync in write timing 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: File size set to 1048576 kB 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Record Size 1024 kB 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Output is in kBytes/sec 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache line size set to 32 bytes. 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: File stride size set to 17 * record size. 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Throughput test with 1 process 2024-09-21T01:24:25.483 INFO:tasks.workunit.client.0.smithi027.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-21T01:24:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:26 smithi098 ceph-mon[34372]: pgmap v460: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 60 MiB/s wr, 24 op/s 2024-09-21T01:24:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:26 smithi027 ceph-mon[30375]: pgmap v460: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 60 MiB/s wr, 24 op/s 2024-09-21T01:24:26.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:26 smithi089 ceph-mon[34486]: pgmap v460: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 60 MiB/s wr, 24 op/s 2024-09-21T01:24:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:28 smithi098 ceph-mon[34372]: pgmap v461: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T01:24:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:28 smithi027 ceph-mon[30375]: pgmap v461: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T01:24:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:28 smithi089 ceph-mon[34486]: pgmap v461: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 29 MiB/s wr, 19 op/s 2024-09-21T01:24:30.215 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:30.216 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 initial writers = 223274.72 kB/sec 2024-09-21T01:24:30.216 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 initial writers = 223238.55 kB/sec 2024-09-21T01:24:30.216 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 223274.72 kB/sec 2024-09-21T01:24:30.216 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 223274.72 kB/sec 2024-09-21T01:24:30.216 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 223274.72 kB/sec 2024-09-21T01:24:30.216 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:30 smithi098 ceph-mon[34372]: pgmap v462: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-09-21T01:24:30.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:30 smithi027 ceph-mon[30375]: pgmap v462: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-09-21T01:24:30.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:30 smithi089 ceph-mon[34486]: pgmap v462: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-09-21T01:24:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:32 smithi098 ceph-mon[34372]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 MiB/s wr, 46 op/s 2024-09-21T01:24:32.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:32 smithi027 ceph-mon[30375]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 MiB/s wr, 46 op/s 2024-09-21T01:24:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:32 smithi089 ceph-mon[34486]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 40 MiB/s wr, 46 op/s 2024-09-21T01:24:34.302 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:34 smithi089 ceph-mon[34486]: pgmap v464: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 71 MiB/s wr, 50 op/s 2024-09-21T01:24:34.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:34 smithi098 ceph-mon[34372]: pgmap v464: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 71 MiB/s wr, 50 op/s 2024-09-21T01:24:34.433 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:34 smithi027 ceph-mon[30375]: pgmap v464: 129 pgs: 129 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 71 MiB/s wr, 50 op/s 2024-09-21T01:24:36.324 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:36 smithi027 ceph-mon[30375]: pgmap v465: 129 pgs: 129 active+clean; 2.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 86 MiB/s wr, 55 op/s 2024-09-21T01:24:36.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:36 smithi098 ceph-mon[34372]: pgmap v465: 129 pgs: 129 active+clean; 2.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 86 MiB/s wr, 55 op/s 2024-09-21T01:24:36.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:36 smithi089 ceph-mon[34486]: pgmap v465: 129 pgs: 129 active+clean; 2.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 86 MiB/s wr, 55 op/s 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 rewriters = 220920.61 kB/sec 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 rewriters = 220885.38 kB/sec 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 220920.61 kB/sec 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 220920.61 kB/sec 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 220920.61 kB/sec 2024-09-21T01:24:37.654 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:38.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:38 smithi098 ceph-mon[34372]: pgmap v466: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 98 MiB/s wr, 60 op/s 2024-09-21T01:24:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:38 smithi027 ceph-mon[30375]: pgmap v466: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 98 MiB/s wr, 60 op/s 2024-09-21T01:24:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:38 smithi089 ceph-mon[34486]: pgmap v466: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 98 MiB/s wr, 60 op/s 2024-09-21T01:24:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:40 smithi098 ceph-mon[34372]: pgmap v467: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 135 MiB/s wr, 66 op/s 2024-09-21T01:24:40.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:40 smithi027 ceph-mon[30375]: pgmap v467: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 135 MiB/s wr, 66 op/s 2024-09-21T01:24:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:40 smithi089 ceph-mon[34486]: pgmap v467: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 135 MiB/s wr, 66 op/s 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 readers = 4736284.50 kB/sec 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 readers = 4719421.13 kB/sec 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 4736284.50 kB/sec 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 4736284.50 kB/sec 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 4736284.50 kB/sec 2024-09-21T01:24:40.838 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:41.380 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T01:24:41.380 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T01:24:41.381 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:24:41.770 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:24:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:42 smithi027 ceph-mon[30375]: pgmap v468: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 160 MiB/s wr, 66 op/s 2024-09-21T01:24:42.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:42 smithi098 ceph-mon[34372]: pgmap v468: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 160 MiB/s wr, 66 op/s 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.451+0000 7fe041200640 1 -- 172.21.15.27:0/191068151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 msgr2=0x7fe03c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.451+0000 7fe041200640 1 --2- 172.21.15.27:0/191068151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c106190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009860 tx=0x7fe02c02a340 comp rx=0 tx=0).stop 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 -- 172.21.15.27:0/191068151 shutdown_connections 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 --2- 172.21.15.27:0/191068151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe03c10c010 0x7fe03c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 --2- 172.21.15.27:0/191068151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe03c10b6c0 0x7fe03c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 --2- 172.21.15.27:0/191068151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c106190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 -- 172.21.15.27:0/191068151 >> 172.21.15.27:0/191068151 conn(0x7fe03c101810 msgr2=0x7fe03c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 -- 172.21.15.27:0/191068151 shutdown_connections 2024-09-21T01:24:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.452+0000 7fe041200640 1 -- 172.21.15.27:0/191068151 wait complete. 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 Processor -- start 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 -- start start 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe03c10b6c0 0x7fe03c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe03c10c010 0x7fe03c1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c115cd0 con 0x7fe03c10c010 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c115b50 con 0x7fe03c10b6c0 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.453+0000 7fe041200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c115e50 con 0x7fe03c105d80 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe03a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe03c10c010 0x7fe03c1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:42.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:59008/0 (socket says 172.21.15.27:59008) 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 -- 172.21.15.27:0/2189440439 learned_addr learned my addr 172.21.15.27:0/2189440439 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe033e00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe03c10b6c0 0x7fe03c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 -- 172.21.15.27:0/2189440439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe03c10b6c0 msgr2=0x7fe03c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe03c10b6c0 0x7fe03c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 -- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe03c10c010 msgr2=0x7fe03c1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe03c10c010 0x7fe03c1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.454+0000 7fe039e00640 1 -- 172.21.15.27:0/2189440439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c1a2610 con 0x7fe03c105d80 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.455+0000 7fe039e00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c19c6a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0303f0 tx=0x7fe02c032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.455+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c040020 con 0x7fe03c105d80 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.455+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c1a28a0 con 0x7fe03c105d80 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.455+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a9060 con 0x7fe03c105d80 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.455+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02c030dd0 con 0x7fe03c105d80 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.455+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c037040 con 0x7fe03c105d80 2024-09-21T01:24:42.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.456+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0180052c0 con 0x7fe03c105d80 2024-09-21T01:24:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.459+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fe02c043020 con 0x7fe03c105d80 2024-09-21T01:24:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.460+0000 7fe032a00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0200b6680 0x7fe0200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.461+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe02c107740 con 0x7fe03c105d80 2024-09-21T01:24:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.461+0000 7fe033e00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0200b6680 0x7fe0200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:42.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.461+0000 7fe033e00640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0200b6680 0x7fe0200b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe03c10b150 tx=0x7fe024009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:42.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.462+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe02c04a070 con 0x7fe03c105d80 2024-09-21T01:24:42.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:42 smithi089 ceph-mon[34486]: pgmap v468: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 160 MiB/s wr, 66 op/s 2024-09-21T01:24:42.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.701+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0180055b0 con 0x7fe03c105d80 2024-09-21T01:24:42.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.702+0000 7fe032a00640 1 -- 172.21.15.27:0/2189440439 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fe02c0328c0 con 0x7fe03c105d80 2024-09-21T01:24:42.703 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:24:42.704 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:24:42.704 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0200b6680 msgr2=0x7fe0200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0200b6680 0x7fe0200b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe03c10b150 tx=0x7fe024009260 comp rx=0 tx=0).stop 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 msgr2=0x7fe03c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c19c6a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0303f0 tx=0x7fe02c032da0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 shutdown_connections 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe0200b6680 0x7fe0200b8b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe03c10c010 0x7fe03c1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe03c10b6c0 0x7fe03c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 --2- 172.21.15.27:0/2189440439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe03c105d80 0x7fe03c19c6a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 >> 172.21.15.27:0/2189440439 conn(0x7fe03c101810 msgr2=0x7fe03c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 shutdown_connections 2024-09-21T01:24:42.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:42.705+0000 7fe041200640 1 -- 172.21.15.27:0/2189440439 wait complete. 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 re-readers = 3389195.75 kB/sec 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 re-readers = 3379736.07 kB/sec 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 3389195.75 kB/sec 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 3389195.75 kB/sec 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 3389195.75 kB/sec 2024-09-21T01:24:43.260 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 1048576.00 kB 2024-09-21T01:24:43.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:43 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2189440439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:24:43.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:43 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2189440439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:24:43.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:43 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2189440439' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:24:44.029 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T01:24:44.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T01:24:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:44 smithi098 ceph-mon[34372]: pgmap v469: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 130 MiB/s wr, 42 op/s 2024-09-21T01:24:44.419 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:24:44.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:44 smithi027 ceph-mon[30375]: pgmap v469: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 130 MiB/s wr, 42 op/s 2024-09-21T01:24:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:44 smithi089 ceph-mon[34486]: pgmap v469: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 130 MiB/s wr, 42 op/s 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3206062795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0077940 msgr2=0x7fd5a0077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3206062795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0077940 0x7fd5a0077d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd594009900 tx=0x7fd59402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3206062795 shutdown_connections 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3206062795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0114740 0x7fd5a0116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3206062795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a0078290 0x7fd5a0114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3206062795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0077940 0x7fd5a0077d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3206062795 >> 172.21.15.27:0/3206062795 conn(0x7fd5a01017d0 msgr2=0x7fd5a0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3206062795 shutdown_connections 2024-09-21T01:24:45.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.216+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3206062795 wait complete. 2024-09-21T01:24:45.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.217+0000 7fd5a7a00640 1 Processor -- start 2024-09-21T01:24:45.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.217+0000 7fd5a7a00640 1 -- start start 2024-09-21T01:24:45.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a7a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 0x7fd5a01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a7a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a0078290 0x7fd5a01a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a7a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0114740 0x7fd5a01a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a7a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a011a0d0 con 0x7fd5a0077940 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a7a00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0119f50 con 0x7fd5a0114740 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a7a00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a011a250 con 0x7fd5a0078290 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 0x7fd5a01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 0x7fd5a01a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38954/0 (socket says 172.21.15.27:38954) 2024-09-21T01:24:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 -- 172.21.15.27:0/3010016617 learned_addr learned my addr 172.21.15.27:0/3010016617 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0114740 0x7fd5a01a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd59fe00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a0078290 0x7fd5a01a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 -- 172.21.15.27:0/3010016617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a0078290 msgr2=0x7fd5a01a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a0078290 0x7fd5a01a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 -- 172.21.15.27:0/3010016617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0114740 msgr2=0x7fd5a01a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.218+0000 7fd5a5000640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0114740 0x7fd5a01a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.219+0000 7fd5a5000640 1 -- 172.21.15.27:0/3010016617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a01a6a30 con 0x7fd5a0077940 2024-09-21T01:24:45.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.219+0000 7fd5a5000640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 0x7fd5a01a0ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd594005940 tx=0x7fd5940040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.219+0000 7fd59d600640 1 -- 172.21.15.27:0/3010016617 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59403f030 con 0x7fd5a0077940 2024-09-21T01:24:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.219+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a01a6cc0 con 0x7fd5a0077940 2024-09-21T01:24:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.219+0000 7fd59d600640 1 -- 172.21.15.27:0/3010016617 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5940044c0 con 0x7fd5a0077940 2024-09-21T01:24:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.219+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01ad480 con 0x7fd5a0077940 2024-09-21T01:24:45.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.220+0000 7fd59d600640 1 -- 172.21.15.27:0/3010016617 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594049060 con 0x7fd5a0077940 2024-09-21T01:24:45.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 Processor -- start 2024-09-21T01:24:45.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 -- start start 2024-09-21T01:24:45.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd59d600640 1 -- 172.21.15.27:0/3010016617 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fd594042030 con 0x7fd5a0077940 2024-09-21T01:24:45.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198322350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f198322890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f198322dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f198297ca0 con 0x55f197f51980 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f1982870d0 con 0x55f197f59b60 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.223+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f19828aee0 con 0x55f197f7a840 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f198322890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198322350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f198322dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198322350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:59044/0 (socket says 172.21.15.27:59044) 2024-09-21T01:24:45.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 -- 172.21.15.27:0/4042591746 learned_addr learned my addr 172.21.15.27:0/4042591746 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 -- 172.21.15.27:0/4042591746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 msgr2=0x55f198322dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f198322dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 -- 172.21.15.27:0/4042591746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 msgr2=0x55f198322890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f198322890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.224+0000 7fd5a5a00640 1 -- 172.21.15.27:0/4042591746 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f198239400 con 0x55f197f7a840 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198322350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd598009fd0 tx=0x7fd5980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a1b278d51603a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:45.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd59d600640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5880b6680 0x7fd5880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd592000640 1 -- 172.21.15.27:0/4042591746 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598004d20 con 0x55f197f7a840 2024-09-21T01:24:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd592000640 1 -- 172.21.15.27:0/4042591746 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd598004050 con 0x55f197f7a840 2024-09-21T01:24:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd592000640 1 -- 172.21.15.27:0/4042591746 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598005260 con 0x55f197f7a840 2024-09-21T01:24:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd5a5000640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5880b6680 0x7fd5880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.225+0000 7fd59d600640 1 -- 172.21.15.27:0/3010016617 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd594107e30 con 0x7fd5a0077940 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/4042591746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 msgr2=0x55f198322350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198322350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd598009fd0 tx=0x7fd5980118d0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5a5000640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5880b6680 0x7fd5880b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f198320fd0 tx=0x7fd59410b8e0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/4042591746 shutdown_connections 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f198322dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f198322890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/4042591746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198322350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/4042591746 >> 172.21.15.27:0/4042591746 conn(0x55f197f6e990 msgr2=0x55f198320240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/4042591746 shutdown_connections 2024-09-21T01:24:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.226+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/4042591746 wait complete. 2024-09-21T01:24:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.227+0000 7fd5ae5c0740 1 -- start start 2024-09-21T01:24:45.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.227+0000 7fd5ae5c0740 1 Processor -- start 2024-09-21T01:24:45.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f19832d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f19832df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f19832e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f1980e87a0 con 0x55f197f51980 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f19822ca10 con 0x55f197f59b60 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f198330b30 con 0x55f197f7a840 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5ae5c0740 10 client.0 _ll_register_callbacks cb 0x55f19823cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f19832d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f19832d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38966/0 (socket says 172.21.15.27:38966) 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd5a5a00640 1 -- 172.21.15.27:0/1491361573 learned_addr learned my addr 172.21.15.27:0/1491361573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.228+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f19832df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f19832e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd59fe00640 1 -- 172.21.15.27:0/1491361573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 msgr2=0x55f19832e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f19832e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd59fe00640 1 -- 172.21.15.27:0/1491361573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 msgr2=0x55f19832d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f19832d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd59fe00640 1 -- 172.21.15.27:0/1491361573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f197ff4b90 con 0x55f197f59b60 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f19832e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f19832df00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5a006c790 tx=0x7fd58c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87c4e83d81c4f116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd586a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd586a00640 1 -- 172.21.15.27:0/1491361573 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004020 con 0x55f197f59b60 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd586a00640 1 -- 172.21.15.27:0/1491361573 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd58c0041c0 con 0x55f197f59b60 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.229+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd598004050 con 0x55f197f59b60 2024-09-21T01:24:45.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.230+0000 7fd586a00640 1 -- 172.21.15.27:0/1491361573 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd58c025050 con 0x55f197f59b60 2024-09-21T01:24:45.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.230+0000 7fd5ae5c0740 10 client.25745 fetch_fsmap learned FSMap version 29 2024-09-21T01:24:45.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.230+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f19833c870 con 0x55f197f59b60 2024-09-21T01:24:45.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.230+0000 7fd586a00640 1 -- 172.21.15.27:0/1491361573 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd598004050 con 0x55f197f59b60 2024-09-21T01:24:45.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.230+0000 7fd586a00640 1 -- 172.21.15.27:0/1491361573 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd58c0037f0 con 0x55f197f59b60 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5ae5c0740 10 client.25745 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5ae5c0740 10 client.25745 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5ae5c0740 20 client.25745 populate_metadata read hostname 'smithi027' 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f391c0 0x55f19833fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5ae5c0740 4 client.25745 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f198299bf0 con 0x55f197f391c0 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.231+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f391c0 0x55f19833fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.232+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f391c0 0x55f19833fec0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf6b59c7e41f0e0 server_cookie=e4ae93936ffedfd8 in_seq=0 out_seq=0 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.232+0000 7fd586a00640 10 client.25745 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.232+0000 7fd586a00640 1 -- 172.21.15.27:0/1491361573 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f198299bf0 con 0x55f197f391c0 2024-09-21T01:24:45.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.232+0000 7fd586a00640 10 client.25745 handle_command_reply: tid=0 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 client.25745 shutdown 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 msgr2=0x55f19832df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f19832df00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5a006c790 tx=0x7fd58c012b60 comp rx=0 tx=0).stop 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 shutdown_connections 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f19832e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f391c0 0x55f19833fec0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f19832df00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1491361573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f51980 0x55f19832d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 >> 172.21.15.27:0/1491361573 conn(0x55f197f6e990 msgr2=0x55f1983254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd586a00640 0 client.25745 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 shutdown_connections 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1491361573 wait complete. 2024-09-21T01:24:45.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.233+0000 7fd5ae5c0740 20 client.25745 trim_cache size 0 max 16384 2024-09-21T01:24:45.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.251+0000 7fd5ae5c0740 1 Processor -- start 2024-09-21T01:24:45.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.251+0000 7fd5ae5c0740 1 -- start start 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f51980 0x55f1983239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 0x55f19832f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198325b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b73f0 con 0x55f197f59b60 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b8f30 con 0x55f197f51980 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b7ac0 con 0x55f197f7a840 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 0x55f19832f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 0x55f19832f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38974/0 (socket says 172.21.15.27:38974) 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd59fe00640 1 -- 172.21.15.27:0/3455603132 learned_addr learned my addr 172.21.15.27:0/3455603132 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f51980 0x55f1983239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.254 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.252+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198325b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd59fe00640 1 -- 172.21.15.27:0/3455603132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 msgr2=0x55f198325b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd59fe00640 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198325b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd59fe00640 1 -- 172.21.15.27:0/3455603132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f51980 msgr2=0x55f1983239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd59fe00640 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f51980 0x55f1983239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd59fe00640 1 -- 172.21.15.27:0/3455603132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19831bc50 con 0x55f197f59b60 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198325b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd59fe00640 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 0x55f19832f8a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd58c001eb0 tx=0x7fd58c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d8f66656d01261 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd591600640 1 -- 172.21.15.27:0/3455603132 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0035f0 con 0x55f197f59b60 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd591600640 1 -- 172.21.15.27:0/3455603132 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58c00fda0 con 0x55f197f59b60 2024-09-21T01:24:45.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.253+0000 7fd591600640 1 -- 172.21.15.27:0/3455603132 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c005420 con 0x55f197f59b60 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/3455603132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 msgr2=0x55f19832f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 0x55f19832f8a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd58c001eb0 tx=0x7fd58c0096c0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/3455603132 shutdown_connections 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f7a840 0x55f198325b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f59b60 0x55f19832f8a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/3455603132 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f51980 0x55f1983239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/3455603132 >> 172.21.15.27:0/3455603132 conn(0x55f197f6e990 msgr2=0x55f198325450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/3455603132 shutdown_connections 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.254+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/3455603132 wait complete. 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.255+0000 7fd5ae5c0740 1 -- start start 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.255+0000 7fd5ae5c0740 1 Processor -- start 2024-09-21T01:24:45.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f51980 0x55f198333ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f1983343e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 0x55f198334920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b8860 con 0x55f197f7a840 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b8190 con 0x55f197f59b60 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b53d0 con 0x55f197f51980 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5ae5c0740 10 client.0 _ll_register_callbacks cb 0x7fd58c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 0x55f198334920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5a5a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 0x55f198334920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38984/0 (socket says 172.21.15.27:38984) 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5a5a00640 1 -- 172.21.15.27:0/1907359238 learned_addr learned my addr 172.21.15.27:0/1907359238 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f1983343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.256+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f51980 0x55f198333ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd59fe00640 1 -- 172.21.15.27:0/1907359238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f51980 msgr2=0x55f198333ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f51980 0x55f198333ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd59fe00640 1 -- 172.21.15.27:0/1907359238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 msgr2=0x55f198334920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 0x55f198334920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd59fe00640 1 -- 172.21.15.27:0/1907359238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1983a9960 con 0x55f197f59b60 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 0x55f198334920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd59fe00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f1983343e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd58c003db0 tx=0x7fd58c012660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52a242ffeb31b95c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd586a00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd586a00640 1 -- 172.21.15.27:0/1907359238 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c022040 con 0x55f197f59b60 2024-09-21T01:24:45.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd586a00640 1 -- 172.21.15.27:0/1907359238 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd58c00fb00 con 0x55f197f59b60 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.257+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c00fda0 con 0x55f197f59b60 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd586a00640 1 -- 172.21.15.27:0/1907359238 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd58c039050 con 0x55f197f59b60 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 10 client.25751 fetch_fsmap learned FSMap version 29 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1983aabc0 con 0x55f197f59b60 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd586a00640 1 -- 172.21.15.27:0/1907359238 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58c00fda0 con 0x55f197f59b60 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd586a00640 1 -- 172.21.15.27:0/1907359238 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd58c01c880 con 0x55f197f59b60 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 10 client.25751 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 10 client.25751 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 20 client.25751 populate_metadata read hostname 'smithi027' 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f0e570 0x55f1983ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 4 client.25751 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.258+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55f1982956a0 con 0x55f197f0e570 2024-09-21T01:24:45.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.259+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f0e570 0x55f1983ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:24:45.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.259+0000 7fd5a5a00640 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f0e570 0x55f1983ad350 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53ce26a0b797e54 server_cookie=b24ffdc4f9fdee0a in_seq=0 out_seq=0 2024-09-21T01:24:45.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.259+0000 7fd586a00640 10 client.25751 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:45.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd586a00640 1 -- 172.21.15.27:0/1907359238 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f1982956a0 con 0x55f197f0e570 2024-09-21T01:24:45.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd586a00640 10 client.25751 handle_command_reply: tid=0 2024-09-21T01:24:45.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 client.25751 shutdown 2024-09-21T01:24:45.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 msgr2=0x55f1983343e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f1983343e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd58c003db0 tx=0x7fd58c012660 comp rx=0 tx=0).stop 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 shutdown_connections 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f197f7a840 0x55f198334920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f197f0e570 0x55f1983ad350 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f197f59b60 0x55f1983343e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 --2- 172.21.15.27:0/1907359238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f197f51980 0x55f198333ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 >> 172.21.15.27:0/1907359238 conn(0x55f197f6e990 msgr2=0x55f19832ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.273+0000 7fd586a00640 0 client.25751 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 shutdown_connections 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5ae5c0740 1 -- 172.21.15.27:0/1907359238 wait complete. 2024-09-21T01:24:45.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5ae5c0740 20 client.25751 trim_cache size 0 max 16384 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 10, 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "2e0024f4": { 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 10, 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:24:45.276 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:24:45.277 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:24:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5880b6680 msgr2=0x7fd5880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5880b6680 0x7fd5880b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f198320fd0 tx=0x7fd59410b8e0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 msgr2=0x7fd5a01a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:24:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.274+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 0x7fd5a01a0ac0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd594005940 tx=0x7fd5940040c0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 shutdown_connections 2024-09-21T01:24:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fd5880b6680 0x7fd5880b8b90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd5a0114740 0x7fd5a01a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5a0078290 0x7fd5a01a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 --2- 172.21.15.27:0/3010016617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5a0077940 0x7fd5a01a0ac0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:24:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 >> 172.21.15.27:0/3010016617 conn(0x7fd5a01017d0 msgr2=0x7fd5a0116870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:24:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 shutdown_connections 2024-09-21T01:24:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:24:45.275+0000 7fd5a7a00640 1 -- 172.21.15.27:0/3010016617 wait complete. 2024-09-21T01:24:46.232 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2e0024f4', quiesce took 4.9 seconds. Will release after: 22.9 2024-09-21T01:24:46.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:46 smithi098 ceph-mon[34372]: pgmap v470: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 99 MiB/s wr, 35 op/s 2024-09-21T01:24:46.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:46 smithi027 ceph-mon[30375]: pgmap v470: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 99 MiB/s wr, 35 op/s 2024-09-21T01:24:46.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:46 smithi089 ceph-mon[34486]: pgmap v470: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 99 MiB/s wr, 35 op/s 2024-09-21T01:24:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:48 smithi098 ceph-mon[34372]: pgmap v471: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-21T01:24:48.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:48 smithi027 ceph-mon[30375]: pgmap v471: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-21T01:24:48.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:48 smithi089 ceph-mon[34486]: pgmap v471: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 85 MiB/s wr, 33 op/s 2024-09-21T01:24:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:50 smithi098 ceph-mon[34372]: pgmap v472: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 73 MiB/s wr, 27 op/s 2024-09-21T01:24:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:50 smithi027 ceph-mon[30375]: pgmap v472: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 73 MiB/s wr, 27 op/s 2024-09-21T01:24:50.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:50 smithi089 ceph-mon[34486]: pgmap v472: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 73 MiB/s wr, 27 op/s 2024-09-21T01:24:52.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:52 smithi098 ceph-mon[34372]: pgmap v473: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 36 MiB/s wr, 22 op/s 2024-09-21T01:24:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:52 smithi027 ceph-mon[30375]: pgmap v473: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 36 MiB/s wr, 22 op/s 2024-09-21T01:24:52.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:52 smithi089 ceph-mon[34486]: pgmap v473: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 36 MiB/s wr, 22 op/s 2024-09-21T01:24:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:54 smithi098 ceph-mon[34372]: pgmap v474: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-09-21T01:24:54.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:54 smithi027 ceph-mon[30375]: pgmap v474: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-09-21T01:24:54.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:54 smithi089 ceph-mon[34486]: pgmap v474: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 10 op/s 2024-09-21T01:24:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:56 smithi098 ceph-mon[34372]: pgmap v475: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T01:24:56.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:56 smithi027 ceph-mon[30375]: pgmap v475: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T01:24:56.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:56 smithi089 ceph-mon[34486]: pgmap v475: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-21T01:24:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:24:58 smithi098 ceph-mon[34372]: pgmap v476: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T01:24:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:24:58 smithi027 ceph-mon[30375]: pgmap v476: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T01:24:58.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:24:58 smithi089 ceph-mon[34486]: pgmap v476: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T01:25:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:00 smithi098 ceph-mon[34372]: pgmap v477: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T01:25:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:00 smithi027 ceph-mon[30375]: pgmap v477: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T01:25:00.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:00 smithi089 ceph-mon[34486]: pgmap v477: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T01:25:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:02 smithi098 ceph-mon[34372]: pgmap v478: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T01:25:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:02 smithi027 ceph-mon[30375]: pgmap v478: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T01:25:02.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:02 smithi089 ceph-mon[34486]: pgmap v478: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T01:25:04.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:04 smithi098 ceph-mon[34372]: pgmap v479: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:25:04.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:04 smithi027 ceph-mon[30375]: pgmap v479: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:25:04.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:04 smithi089 ceph-mon[34486]: pgmap v479: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:25:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:06 smithi098 ceph-mon[34372]: pgmap v480: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:06 smithi027 ceph-mon[30375]: pgmap v480: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:06.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:06 smithi089 ceph-mon[34486]: pgmap v480: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:08.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:08 smithi098 ceph-mon[34372]: pgmap v481: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T01:25:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:08 smithi027 ceph-mon[30375]: pgmap v481: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T01:25:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:08 smithi089 ceph-mon[34486]: pgmap v481: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T01:25:09.134 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:25:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:25:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 -- 172.21.15.27:0/4217468366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 msgr2=0x7f7da010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 --2- 172.21.15.27:0/4217468366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 0x7f7da010fbe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009900 tx=0x7f7d8c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 -- 172.21.15.27:0/4217468366 shutdown_connections 2024-09-21T01:25:10.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 --2- 172.21.15.27:0/4217468366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0110120 0x7f7da0112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 --2- 172.21.15.27:0/4217468366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 0x7f7da010fbe0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 --2- 172.21.15.27:0/4217468366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7da0075750 0x7f7da0075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 -- 172.21.15.27:0/4217468366 >> 172.21.15.27:0/4217468366 conn(0x7f7da007b850 msgr2=0x7f7da007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 -- 172.21.15.27:0/4217468366 shutdown_connections 2024-09-21T01:25:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.221+0000 7f7da6200640 1 -- 172.21.15.27:0/4217468366 wait complete. 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.222+0000 7f7da6200640 1 Processor -- start 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.222+0000 7f7da6200640 1 -- start start 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7da6200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 0x7f7da019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7da6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 0x7f7da019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7da6200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7da0110120 0x7f7da01a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7da6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0115ce0 con 0x7f7da00760a0 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7da6200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0115b60 con 0x7f7da0110120 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7da6200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0115e60 con 0x7f7da0075750 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 0x7f7da019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 0x7f7da019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 0x7f7da019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:37626/0 (socket says 172.21.15.27:37626) 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 -- 172.21.15.27:0/3160078998 learned_addr learned my addr 172.21.15.27:0/3160078998 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9fe00640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7da0110120 0x7f7da01a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 -- 172.21.15.27:0/3160078998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7da0110120 msgr2=0x7f7da01a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7da0110120 0x7f7da01a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 -- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 msgr2=0x7f7da019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 0x7f7da019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.223+0000 7f7d9f400640 1 -- 172.21.15.27:0/3160078998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da01a2620 con 0x7f7da0075750 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.224+0000 7f7d9f400640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 0x7f7da019c6b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000e9f0 tx=0x7f7d9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.224+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000cd80 con 0x7f7da0075750 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.224+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d900108e0 con 0x7f7da0075750 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.224+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da01a2910 con 0x7f7da0075750 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.224+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90018980 con 0x7f7da0075750 2024-09-21T01:25:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.224+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01a9070 con 0x7f7da0075750 2024-09-21T01:25:10.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.225+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da0075b60 con 0x7f7da0075750 2024-09-21T01:25:10.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.228+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7d90010430 con 0x7f7da0075750 2024-09-21T01:25:10.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.229+0000 7f7d97e00640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7d840b6680 0x7f7d840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:10.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.230+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7d900e8220 con 0x7f7da0075750 2024-09-21T01:25:10.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.230+0000 7f7d9ea00640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7d840b6680 0x7f7d840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:10.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.231+0000 7f7d9ea00640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7d840b6680 0x7f7d840b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c02a9e0 tx=0x7f7d8c005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:10.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.231+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7d900b2780 con 0x7f7da0075750 2024-09-21T01:25:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:10 smithi098 ceph-mon[34372]: pgmap v482: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:10 smithi027 ceph-mon[30375]: pgmap v482: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:10.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.473+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da019d7e0 con 0x7f7da0075750 2024-09-21T01:25:10.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.474+0000 7f7d97e00640 1 -- 172.21.15.27:0/3160078998 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f7d900b76c0 con 0x7f7da0075750 2024-09-21T01:25:10.475 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:25:10.475 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:25:10.475 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:25:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7d840b6680 msgr2=0x7f7d840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7d840b6680 0x7f7d840b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c02a9e0 tx=0x7f7d8c005a20 comp rx=0 tx=0).stop 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 msgr2=0x7f7da019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 0x7f7da019c6b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000e9f0 tx=0x7f7d9000eeb0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 shutdown_connections 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f7d840b6680 0x7f7d840b8b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7da0110120 0x7f7da01a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da00760a0 0x7f7da019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 --2- 172.21.15.27:0/3160078998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da0075750 0x7f7da019c6b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 >> 172.21.15.27:0/3160078998 conn(0x7f7da007b850 msgr2=0x7f7da0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 shutdown_connections 2024-09-21T01:25:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:10.477+0000 7f7da6200640 1 -- 172.21.15.27:0/3160078998 wait complete. 2024-09-21T01:25:10.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:10 smithi089 ceph-mon[34486]: pgmap v482: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:11 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3160078998' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:11.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:11 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3160078998' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:11.449 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db --set-id 2e0024f4 --release --await' 2024-09-21T01:25:11.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db --set-id 2e0024f4 --release --await 2024-09-21T01:25:11.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:11 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3160078998' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:25:12.324 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:12 smithi027 ceph-mon[30375]: pgmap v483: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-21T01:25:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:12 smithi098 ceph-mon[34372]: pgmap v483: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-21T01:25:12.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:12 smithi089 ceph-mon[34486]: pgmap v483: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 -- 172.21.15.27:0/3399487917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e41094a0 msgr2=0x7f24e41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 --2- 172.21.15.27:0/3399487917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e41094a0 0x7f24e41098b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009900 tx=0x7f24d002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 -- 172.21.15.27:0/3399487917 shutdown_connections 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 --2- 172.21.15.27:0/3399487917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4109df0 0x7f24e4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 --2- 172.21.15.27:0/3399487917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e41094a0 0x7f24e41098b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 --2- 172.21.15.27:0/3399487917 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e4103b60 0x7f24e4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 -- 172.21.15.27:0/3399487917 >> 172.21.15.27:0/3399487917 conn(0x7f24e407b850 msgr2=0x7f24e407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 -- 172.21.15.27:0/3399487917 shutdown_connections 2024-09-21T01:25:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.515+0000 7f24eb200640 1 -- 172.21.15.27:0/3399487917 wait complete. 2024-09-21T01:25:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.516+0000 7f24eb200640 1 Processor -- start 2024-09-21T01:25:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 -- start start 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 0x7f24e419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e41094a0 0x7f24e419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4109df0 0x7f24e419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4113ab0 con 0x7f24e4103b60 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4113930 con 0x7f24e41094a0 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24eb200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4113c30 con 0x7f24e4109df0 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 0x7f24e419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 0x7f24e419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39866/0 (socket says 172.21.15.27:39866) 2024-09-21T01:25:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24e3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e41094a0 0x7f24e419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.517+0000 7f24e3e00640 1 -- 172.21.15.27:0/1049749159 learned_addr learned my addr 172.21.15.27:0/1049749159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:12.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e9200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4109df0 0x7f24e419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e3e00640 1 -- 172.21.15.27:0/1049749159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4109df0 msgr2=0x7f24e419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e3e00640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4109df0 0x7f24e419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e3e00640 1 -- 172.21.15.27:0/1049749159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e41094a0 msgr2=0x7f24e419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e3e00640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e41094a0 0x7f24e419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e3e00640 1 -- 172.21.15.27:0/1049749159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41a0430 con 0x7f24e4103b60 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e9200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4109df0 0x7f24e419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.518+0000 7f24e3e00640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 0x7f24e419a4c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f24d400e9f0 tx=0x7f24d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.519+0000 7f24e0c00640 1 -- 172.21.15.27:0/1049749159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d400ccc0 con 0x7f24e4103b60 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.519+0000 7f24e0c00640 1 -- 172.21.15.27:0/1049749159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d40108e0 con 0x7f24e4103b60 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.519+0000 7f24e0c00640 1 -- 172.21.15.27:0/1049749159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4018880 con 0x7f24e4103b60 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.519+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e41a0720 con 0x7f24e4103b60 2024-09-21T01:25:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.519+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e41a6e80 con 0x7f24e4103b60 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.522+0000 7f24f1e2b740 1 Processor -- start 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.522+0000 7f24f1e2b740 1 -- start start 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a3e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ce39b5290 con 0x563ce366e660 2024-09-21T01:25:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563ce39a46c0 con 0x563ce3697520 2024-09-21T01:25:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563ce39a84d0 con 0x563ce3676840 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e0c00640 1 -- 172.21.15.27:0/1049749159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f24d4010430 con 0x7f24e4103b60 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a3e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e9200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a3e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38912/0 (socket says 172.21.15.27:38912) 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e9200640 1 -- 172.21.15.27:0/437360542 learned_addr learned my addr 172.21.15.27:0/437360542 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e3e00640 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.523+0000 7f24e9200640 1 -- 172.21.15.27:0/437360542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 msgr2=0x563ce3a3ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24e9200640 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a3ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24e9200640 1 -- 172.21.15.27:0/437360542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 msgr2=0x563ce3a3e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24e9200640 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a3e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24e9200640 1 -- 172.21.15.27:0/437360542 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce3931460 con 0x563ce3697520 2024-09-21T01:25:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24e3e00640 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:25:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24e9200640 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a3e1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24dc011810 tx=0x7f24dc011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e077cb162fb3d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24d9600640 1 -- 172.21.15.27:0/437360542 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc011310 con 0x563ce3697520 2024-09-21T01:25:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24d9600640 1 -- 172.21.15.27:0/437360542 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc00f9a0 con 0x563ce3697520 2024-09-21T01:25:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.524+0000 7f24d9600640 1 -- 172.21.15.27:0/437360542 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc005650 con 0x563ce3697520 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 -- 172.21.15.27:0/437360542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 msgr2=0x563ce3a3e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a3e1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24dc011810 tx=0x7f24dc011cd0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24e0c00640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24c80b6680 0x7f24c80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 -- 172.21.15.27:0/437360542 shutdown_connections 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a3ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/437360542 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a3e1c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.525+0000 7f24f1e2b740 1 -- 172.21.15.27:0/437360542 >> 172.21.15.27:0/437360542 conn(0x563ce368b910 msgr2=0x563ce3a3c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.526+0000 7f24e0c00640 1 -- 172.21.15.27:0/1049749159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f24d40e7ef0 con 0x7f24e4103b60 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.526+0000 7f24e9200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24c80b6680 0x7f24c80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.526+0000 7f24f1e2b740 1 -- 172.21.15.27:0/437360542 shutdown_connections 2024-09-21T01:25:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.526+0000 7f24f1e2b740 1 -- 172.21.15.27:0/437360542 wait complete. 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.526+0000 7f24f1e2b740 1 -- start start 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24e9200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24c80b6680 0x7f24c80b8b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005fd0 tx=0x7f24dc000f40 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 Processor -- start 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563ce3805bb0 con 0x563ce366e660 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563ce393f6c0 con 0x563ce3697520 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563ce3a50cd0 con 0x563ce3676840 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.527+0000 7f24f1e2b740 10 client.0 _ll_register_callbacks cb 0x563ce395a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:25:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:37662/0 (socket says 172.21.15.27:37662) 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 -- 172.21.15.27:0/924352506 learned_addr learned my addr 172.21.15.27:0/924352506 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e9200640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 -- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 msgr2=0x563ce3a5a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a5a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 -- 172.21.15.27:0/924352506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 msgr2=0x563ce3a4e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a4e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3400640 1 -- 172.21.15.27:0/924352506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce3702b00 con 0x563ce3676840 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e3e00640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.528+0000 7f24e9200640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24e3400640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a4e680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009900 tx=0x7f24d0032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=597c0a82573d339c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24ce000640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:25:12.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24ce000640 1 -- 172.21.15.27:0/924352506 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0035040 con 0x563ce3676840 2024-09-21T01:25:12.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24dc00f9a0 con 0x563ce3676840 2024-09-21T01:25:12.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24ce000640 1 -- 172.21.15.27:0/924352506 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f24d0031760 con 0x563ce3676840 2024-09-21T01:25:12.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24ce000640 1 -- 172.21.15.27:0/924352506 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24d004f050 con 0x563ce3676840 2024-09-21T01:25:12.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24f1e2b740 10 client.25747 fetch_fsmap learned FSMap version 29 2024-09-21T01:25:12.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.529+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ce3779960 con 0x563ce3676840 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24ce000640 1 -- 172.21.15.27:0/924352506 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc00f9a0 con 0x563ce3676840 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24ce000640 1 -- 172.21.15.27:0/924352506 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f24d0004750 con 0x563ce3676840 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24f1e2b740 10 client.25747 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24f1e2b740 10 client.25747 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24f1e2b740 20 client.25747 populate_metadata read hostname 'smithi027' 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce3655ea0 0x563ce3a5ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24f1e2b740 4 client.25747 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce39b71e0 con 0x563ce3655ea0 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.530+0000 7f24e3e00640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce3655ea0 0x563ce3a5ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.531+0000 7f24e3e00640 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce3655ea0 0x563ce3a5ff70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82f7638273a3204 server_cookie=35bc08ddc0e4e497 in_seq=0 out_seq=0 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.531+0000 7f24ce000640 10 client.25747 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24ce000640 1 -- 172.21.15.27:0/924352506 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ce39b71e0 con 0x563ce3655ea0 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24ce000640 10 client.25747 handle_command_reply: tid=0 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 client.25747 shutdown 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 msgr2=0x563ce3a4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a4e680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009900 tx=0x7f24d0032e90 comp rx=0 tx=0).stop 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 shutdown_connections 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce3655ea0 0x563ce3a5ff70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3697520 0x563ce3a5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3676840 0x563ce3a4e680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/924352506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24ce000640 0 client.25747 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 >> 172.21.15.27:0/924352506 conn(0x563ce368b910 msgr2=0x563ce3a3a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 shutdown_connections 2024-09-21T01:25:12.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 1 -- 172.21.15.27:0/924352506 wait complete. 2024-09-21T01:25:12.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.532+0000 7f24f1e2b740 20 client.25747 trim_cache size 0 max 16384 2024-09-21T01:25:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 Processor -- start 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 -- start start 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce366e660 0x563ce3a4fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce3697520 0x563ce3a5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41aecb0 con 0x563ce3697520 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41b07f0 con 0x563ce3676840 2024-09-21T01:25:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41af380 con 0x563ce366e660 2024-09-21T01:25:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24e9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce3697520 0x563ce3a5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24e9200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce3697520 0x563ce3a5a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39904/0 (socket says 172.21.15.27:39904) 2024-09-21T01:25:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24e9200640 1 -- 172.21.15.27:0/3354075604 learned_addr learned my addr 172.21.15.27:0/3354075604 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.551+0000 7f24e3e00640 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce366e660 0x563ce3a4fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3400640 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3e00640 1 -- 172.21.15.27:0/3354075604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 msgr2=0x563ce3a59f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3e00640 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a59f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3e00640 1 -- 172.21.15.27:0/3354075604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce3697520 msgr2=0x563ce3a5a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3e00640 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce3697520 0x563ce3a5a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3e00640 1 -- 172.21.15.27:0/3354075604 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce3a5a980 con 0x563ce366e660 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24e3e00640 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce366e660 0x563ce3a4fe20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24d40e5270 tx=0x7f24d40a9940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b68d81672e82d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24d8c00640 1 -- 172.21.15.27:0/3354075604 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40ad480 con 0x563ce366e660 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24d8c00640 1 -- 172.21.15.27:0/3354075604 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d40ae070 con 0x563ce366e660 2024-09-21T01:25:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.552+0000 7f24d8c00640 1 -- 172.21.15.27:0/3354075604 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40b2240 con 0x563ce366e660 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 -- 172.21.15.27:0/3354075604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce366e660 msgr2=0x563ce3a4fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce366e660 0x563ce3a4fe20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24d40e5270 tx=0x7f24d40a9940 comp rx=0 tx=0).stop 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 -- 172.21.15.27:0/3354075604 shutdown_connections 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce3697520 0x563ce3a5a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/3354075604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce366e660 0x563ce3a4fe20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 -- 172.21.15.27:0/3354075604 >> 172.21.15.27:0/3354075604 conn(0x563ce368b910 msgr2=0x563ce3a3acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.553+0000 7f24f1e2b740 1 -- 172.21.15.27:0/3354075604 shutdown_connections 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.554+0000 7f24f1e2b740 1 -- 172.21.15.27:0/3354075604 wait complete. 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.554+0000 7f24f1e2b740 1 -- start start 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.554+0000 7f24f1e2b740 1 Processor -- start 2024-09-21T01:25:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a56670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a52b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3697520 0x563ce3a530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41b0120 con 0x563ce366e660 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41afa50 con 0x563ce3676840 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41acc90 con 0x563ce3697520 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24f1e2b740 10 client.0 _ll_register_callbacks cb 0x7f24d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a56670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24e3e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a56670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39908/0 (socket says 172.21.15.27:39908) 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24e3e00640 1 -- 172.21.15.27:0/1038445632 learned_addr learned my addr 172.21.15.27:0/1038445632 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24e9200640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3697520 0x563ce3a530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.555+0000 7f24e3400640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a52b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24e3e00640 1 -- 172.21.15.27:0/1038445632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3697520 msgr2=0x563ce3a530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24e3e00640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3697520 0x563ce3a530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24e3e00640 1 -- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 msgr2=0x563ce3a52b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24e3e00640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a52b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24e3e00640 1 -- 172.21.15.27:0/1038445632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce3a537b0 con 0x563ce366e660 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24e3e00640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a56670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24d40a9f40 tx=0x7f24d40b2140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b44c72d740bb48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24ce000640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24ce000640 1 -- 172.21.15.27:0/1038445632 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40e5700 con 0x563ce366e660 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24ce000640 1 -- 172.21.15.27:0/1038445632 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f24d40a92c0 con 0x563ce366e660 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d40ae070 con 0x563ce366e660 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.556+0000 7f24ce000640 1 -- 172.21.15.27:0/1038445632 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24d40b2220 con 0x563ce366e660 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 10 client.15726 fetch_fsmap learned FSMap version 29 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ce3acb7e0 con 0x563ce366e660 2024-09-21T01:25:12.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24ce000640 1 -- 172.21.15.27:0/1038445632 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d40ae070 con 0x563ce366e660 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24ce000640 1 -- 172.21.15.27:0/1038445632 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f24d40d2070 con 0x563ce366e660 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 10 client.15726 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 10 client.15726 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 20 client.15726 populate_metadata read hostname 'smithi027' 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce362b250 0x563ce3acdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 4 client.15726 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2e0024f4", "release": true, "await": true}] 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.557+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2e0024f4", "release": true, "await": true}) -- 0x563ce39b2c90 con 0x563ce362b250 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.558+0000 7f24e3400640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce362b250 0x563ce3acdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.558+0000 7f24e3400640 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce362b250 0x563ce3acdfd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1932b13839ecbf server_cookie=425c7b1081b64ef0 in_seq=0 out_seq=0 2024-09-21T01:25:12.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.558+0000 7f24ce000640 10 client.15726 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.561+0000 7f24ce000640 1 -- 172.21.15.27:0/1038445632 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563ce39b2c90 con 0x563ce362b250 2024-09-21T01:25:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.561+0000 7f24ce000640 10 client.15726 handle_command_reply: tid=0 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.561+0000 7f24f1e2b740 1 client.15726 shutdown 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 msgr2=0x563ce3a56670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a56670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f24d40a9f40 tx=0x7f24d40b2140 comp rx=0 tx=0).stop 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 shutdown_connections 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563ce3697520 0x563ce3a530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x563ce362b250 0x563ce3acdfd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563ce3676840 0x563ce3a52b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 --2- 172.21.15.27:0/1038445632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563ce366e660 0x563ce3a56670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 >> 172.21.15.27:0/1038445632 conn(0x563ce368b910 msgr2=0x563ce3a4c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24ce000640 0 client.15726 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 shutdown_connections 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 1 -- 172.21.15.27:0/1038445632 wait complete. 2024-09-21T01:25:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.562+0000 7f24f1e2b740 20 client.15726 trim_cache size 0 max 16384 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 12, 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "2e0024f4": { 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 12, 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:25:12.564 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:12.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:12.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:12.566 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:25:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24c80b6680 msgr2=0x7f24c80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24c80b6680 0x7f24c80b8b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24dc005fd0 tx=0x7f24dc000f40 comp rx=0 tx=0).stop 2024-09-21T01:25:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 msgr2=0x7f24e419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 0x7f24e419a4c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f24d400e9f0 tx=0x7f24d400edf0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 shutdown_connections 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f24c80b6680 0x7f24c80b8b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24e4109df0 0x7f24e419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f24e41094a0 0x7f24e419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 --2- 172.21.15.27:0/1049749159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24e4103b60 0x7f24e419a4c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 >> 172.21.15.27:0/1049749159 conn(0x7f24e407b850 msgr2=0x7f24e4105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 shutdown_connections 2024-09-21T01:25:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:12.563+0000 7f24eb200640 1 -- 172.21.15.27:0/1049749159 wait complete. 2024-09-21T01:25:14.006 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2e0024f4', total seconds elapsed: 32.6 2024-09-21T01:25:14.006 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T01:25:14.006 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:25:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:25:14.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:14 smithi098 ceph-mon[34372]: pgmap v484: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:14 smithi027 ceph-mon[30375]: pgmap v484: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:14 smithi089 ceph-mon[34486]: pgmap v484: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:25:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 -- 172.21.15.27:0/1923473043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc69810b6c0 msgr2=0x7fc69810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 --2- 172.21.15.27:0/1923473043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc69810b6c0 0x7fc69810bad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc680009900 tx=0x7fc68002a410 comp rx=0 tx=0).stop 2024-09-21T01:25:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 -- 172.21.15.27:0/1923473043 shutdown_connections 2024-09-21T01:25:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 --2- 172.21.15.27:0/1923473043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810c010 0x7fc698112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 --2- 172.21.15.27:0/1923473043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc69810b6c0 0x7fc69810bad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 --2- 172.21.15.27:0/1923473043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc698105d80 0x7fc698106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 -- 172.21.15.27:0/1923473043 >> 172.21.15.27:0/1923473043 conn(0x7fc698101810 msgr2=0x7fc698103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 -- 172.21.15.27:0/1923473043 shutdown_connections 2024-09-21T01:25:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.054+0000 7fc69e800640 1 -- 172.21.15.27:0/1923473043 wait complete. 2024-09-21T01:25:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.055+0000 7fc69e800640 1 Processor -- start 2024-09-21T01:25:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 -- start start 2024-09-21T01:25:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698105d80 0x7fc69819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810b6c0 0x7fc69819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc69810c010 0x7fc6981a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc698115cd0 con 0x7fc698105d80 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc698115b50 con 0x7fc69810b6c0 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.056+0000 7fc69e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc698115e50 con 0x7fc69810c010 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698105d80 0x7fc69819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698105d80 0x7fc69819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39934/0 (socket says 172.21.15.27:39934) 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc69810c010 0x7fc6981a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697400640 1 -- 172.21.15.27:0/1837911146 learned_addr learned my addr 172.21.15.27:0/1837911146 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc68fe00640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810b6c0 0x7fc69819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 -- 172.21.15.27:0/1837911146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810b6c0 msgr2=0x7fc69819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810b6c0 0x7fc69819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 -- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698105d80 msgr2=0x7fc69819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698105d80 0x7fc69819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 -- 172.21.15.27:0/1837911146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6981a2630 con 0x7fc69810c010 2024-09-21T01:25:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc68fe00640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810b6c0 0x7fc69819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.057+0000 7fc697e00640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc69810c010 0x7fc6981a1f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc69000da10 tx=0x7fc69000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.058+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69000bd70 con 0x7fc69810c010 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.058+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6981a2920 con 0x7fc69810c010 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.058+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc690003e20 con 0x7fc69810c010 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.058+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6900106c0 con 0x7fc69810c010 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.058+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a9080 con 0x7fc69810c010 2024-09-21T01:25:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.059+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc698106190 con 0x7fc69810c010 2024-09-21T01:25:15.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.062+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fc690004270 con 0x7fc69810c010 2024-09-21T01:25:15.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.063+0000 7fc694c00640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc67c0b6680 0x7fc67c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:15.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.063+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fc6900e82c0 con 0x7fc69810c010 2024-09-21T01:25:15.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.064+0000 7fc697400640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc67c0b6680 0x7fc67c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:15.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.064+0000 7fc697400640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc67c0b6680 0x7fc67c0b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc688009cd0 tx=0x7fc688005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.065+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6900ae440 con 0x7fc69810c010 2024-09-21T01:25:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.308+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc69819d7f0 con 0x7fc69810c010 2024-09-21T01:25:15.310 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.309+0000 7fc694c00640 1 -- 172.21.15.27:0/1837911146 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fc69819d7f0 con 0x7fc69810c010 2024-09-21T01:25:15.311 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:25:15.311 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:25:15.311 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:25:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc67c0b6680 msgr2=0x7fc67c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc67c0b6680 0x7fc67c0b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc688009cd0 tx=0x7fc688005f90 comp rx=0 tx=0).stop 2024-09-21T01:25:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc69810c010 msgr2=0x7fc6981a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:15.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc69810c010 0x7fc6981a1f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc69000da10 tx=0x7fc69000ded0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 shutdown_connections 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fc67c0b6680 0x7fc67c0b8b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc69810c010 0x7fc6981a1f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc69810b6c0 0x7fc69819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 --2- 172.21.15.27:0/1837911146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698105d80 0x7fc69819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 >> 172.21.15.27:0/1837911146 conn(0x7fc698101810 msgr2=0x7fc698103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 shutdown_connections 2024-09-21T01:25:15.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:15.311+0000 7fc69e800640 1 -- 172.21.15.27:0/1837911146 wait complete. 2024-09-21T01:25:15.439 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:25:15.439 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:25:15.441 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:25:15.441 INFO:tasks.workunit.client.0.smithi027.stdout:iozone test complete. 2024-09-21T01:25:15.443 INFO:tasks.workunit.client.0.smithi027.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Iozone: Performance Test of File I/O 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Version $Revision: 3.494 $ 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Compiled for 64 bit mode. 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Build: linux-AMD64 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-21T01:25:15.524 INFO:tasks.workunit.client.0.smithi027.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Run began: Sat Sep 21 01:25:15 2024 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Include close in write timing 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Include fsync in write timing 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: File size set to 10485760 kB 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Record Size 1024 kB 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Output is in kBytes/sec 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Time Resolution = 0.000001 seconds. 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache size set to 1024 kBytes. 2024-09-21T01:25:15.525 INFO:tasks.workunit.client.0.smithi027.stdout: Processor cache line size set to 32 bytes. 2024-09-21T01:25:15.526 INFO:tasks.workunit.client.0.smithi027.stdout: File stride size set to 17 * record size. 2024-09-21T01:25:15.526 INFO:tasks.workunit.client.0.smithi027.stdout: Throughput test with 1 process 2024-09-21T01:25:15.526 INFO:tasks.workunit.client.0.smithi027.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-21T01:25:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:16 smithi098 ceph-mon[34372]: pgmap v485: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T01:25:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:16 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1837911146' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:16.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:16 smithi027 ceph-mon[30375]: pgmap v485: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T01:25:16.402 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:16 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1837911146' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:16.405 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T01:25:16.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T01:25:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:16 smithi089 ceph-mon[34486]: pgmap v485: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T01:25:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:16 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1837911146' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:16.809 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:25:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 -- 172.21.15.27:0/3554443005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61781049d0 msgr2=0x7f6178104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 --2- 172.21.15.27:0/3554443005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61781049d0 0x7f6178104de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6168009900 tx=0x7f616802a3f0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 -- 172.21.15.27:0/3554443005 shutdown_connections 2024-09-21T01:25:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 --2- 172.21.15.27:0/3554443005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 0x7f6178110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 --2- 172.21.15.27:0/3554443005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6178105320 0x7f617810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 --2- 172.21.15.27:0/3554443005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f61781049d0 0x7f6178104de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 -- 172.21.15.27:0/3554443005 >> 172.21.15.27:0/3554443005 conn(0x7f6178079760 msgr2=0x7f6178079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 -- 172.21.15.27:0/3554443005 shutdown_connections 2024-09-21T01:25:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.528+0000 7f617e800640 1 -- 172.21.15.27:0/3554443005 wait complete. 2024-09-21T01:25:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 Processor -- start 2024-09-21T01:25:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 -- start start 2024-09-21T01:25:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 0x7f6178102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 0x7f6178102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 0x7f6178103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6178113ae0 con 0x7f617810de30 2024-09-21T01:25:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6178113960 con 0x7f6178105320 2024-09-21T01:25:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.530+0000 7f617e800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6178113c60 con 0x7f61781049d0 2024-09-21T01:25:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 0x7f6178102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6177400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 0x7f6178102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6177400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 0x7f6178102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60122/0 (socket says 172.21.15.27:60122) 2024-09-21T01:25:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 0x7f6178102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54874/0 (socket says 172.21.15.27:54874) 2024-09-21T01:25:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6177400640 1 -- 172.21.15.27:0/1514509751 learned_addr learned my addr 172.21.15.27:0/1514509751 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6177e00640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 0x7f6178103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 -- 172.21.15.27:0/1514509751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 msgr2=0x7f6178102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 0x7f6178102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 -- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 msgr2=0x7f6178103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 0x7f6178103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6176a00640 1 -- 172.21.15.27:0/1514509751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61781a87f0 con 0x7f6178105320 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6177400640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 0x7f6178102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.531+0000 7f6177e00640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 0x7f6178103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:25:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.532+0000 7f6176a00640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 0x7f6178102c50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f617806ab20 tx=0x7f616400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.532+0000 7f616fe00640 1 -- 172.21.15.27:0/1514509751 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400cc40 con 0x7f6178105320 2024-09-21T01:25:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.532+0000 7f616fe00640 1 -- 172.21.15.27:0/1514509751 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616400cde0 con 0x7f6178105320 2024-09-21T01:25:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.532+0000 7f616fe00640 1 -- 172.21.15.27:0/1514509751 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164018780 con 0x7f6178105320 2024-09-21T01:25:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.532+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61781a8ae0 con 0x7f6178105320 2024-09-21T01:25:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.532+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61781a8fd0 con 0x7f6178105320 2024-09-21T01:25:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.537+0000 7f6185403740 1 Processor -- start 2024-09-21T01:25:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.537+0000 7f6185403740 1 -- start start 2024-09-21T01:25:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.537+0000 7f6185403740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 0x55f516e3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6185403740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6185403740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e3add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6185403740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f516dafca0 con 0x55f516a69980 2024-09-21T01:25:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6185403740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f516d9f0d0 con 0x55f516a92840 2024-09-21T01:25:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6185403740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f516da2ee0 con 0x55f516a71b60 2024-09-21T01:25:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6177e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 0x55f516e3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6177e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 0x55f516e3a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54884/0 (socket says 172.21.15.27:54884) 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6177e00640 1 -- 172.21.15.27:0/187089868 learned_addr learned my addr 172.21.15.27:0/187089868 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.538+0000 7f6177400640 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.539+0000 7f6177e00640 1 -- 172.21.15.27:0/187089868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 msgr2=0x55f516e3add0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.539+0000 7f6177e00640 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e3add0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.539+0000 7f6177e00640 1 -- 172.21.15.27:0/187089868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 msgr2=0x55f516e3a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.539+0000 7f6177e00640 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.539+0000 7f6177e00640 1 -- 172.21.15.27:0/187089868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f516d51400 con 0x55f516a92840 2024-09-21T01:25:17.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.539+0000 7f616fe00640 1 -- 172.21.15.27:0/1514509751 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6164010a90 con 0x7f6178105320 2024-09-21T01:25:17.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f6177e00640 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 0x55f516e3a350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6170009fd0 tx=0x7f6170010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b94009c1d183e34a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:17.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f616cc00640 1 -- 172.21.15.27:0/187089868 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170011700 con 0x55f516a92840 2024-09-21T01:25:17.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f616cc00640 1 -- 172.21.15.27:0/187089868 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6170011d50 con 0x55f516a92840 2024-09-21T01:25:17.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f616cc00640 1 -- 172.21.15.27:0/187089868 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617000e3c0 con 0x55f516a92840 2024-09-21T01:25:17.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f616fe00640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f615c0b6680 0x7f615c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f616fe00640 1 -- 172.21.15.27:0/1514509751 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f61640e7e90 con 0x7f6178105320 2024-09-21T01:25:17.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f6177400640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f615c0b6680 0x7f615c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f6185403740 1 -- 172.21.15.27:0/187089868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 msgr2=0x55f516e3a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.541+0000 7f6185403740 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 0x55f516e3a350 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6170009fd0 tx=0x7f6170010860 comp rx=0 tx=0).stop 2024-09-21T01:25:17.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 -- 172.21.15.27:0/187089868 shutdown_connections 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e3add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 --2- 172.21.15.27:0/187089868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a92840 0x55f516e3a350 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 -- 172.21.15.27:0/187089868 >> 172.21.15.27:0/187089868 conn(0x55f516a86990 msgr2=0x55f516e38240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 -- 172.21.15.27:0/187089868 shutdown_connections 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6177400640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f615c0b6680 0x7f615c0b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6168006010 tx=0x7f6168005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:17.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.542+0000 7f6185403740 1 -- 172.21.15.27:0/187089868 wait complete. 2024-09-21T01:25:17.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.543+0000 7f6185403740 1 -- start start 2024-09-21T01:25:17.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 Processor -- start 2024-09-21T01:25:17.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 0x55f516e45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e45fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516e51e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f516c007a0 con 0x55f516a92840 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f516d44a10 con 0x55f516a71b60 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.544+0000 7f6185403740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f516e48b30 con 0x55f516a69980 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6185403740 10 client.0 _ll_register_callbacks cb 0x55f516d54be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 0x55f516e45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 0x55f516e45a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:60126/0 (socket says 172.21.15.27:60126) 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 -- 172.21.15.27:0/2553612186 learned_addr learned my addr 172.21.15.27:0/2553612186 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:17.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 -- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 msgr2=0x55f516e45fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e45fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177e00640 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516e51e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 -- 172.21.15.27:0/2553612186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 msgr2=0x55f516e51e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516e51e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.545+0000 7f6177400640 1 -- 172.21.15.27:0/2553612186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f516b0cb90 con 0x55f516a69980 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.546+0000 7f6177400640 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 0x55f516e45a90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f616802a910 tx=0x7f6168009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c01226c8cf3d404 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:17.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.546+0000 7f6161600640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:25:17.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.546+0000 7f6161600640 1 -- 172.21.15.27:0/2553612186 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168035040 con 0x55f516a69980 2024-09-21T01:25:17.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.546+0000 7f6161600640 1 -- 172.21.15.27:0/2553612186 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6168031bb0 con 0x55f516a69980 2024-09-21T01:25:17.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6170011d50 con 0x55f516a69980 2024-09-21T01:25:17.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6161600640 1 -- 172.21.15.27:0/2553612186 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6168031db0 con 0x55f516a69980 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6185403740 10 client.25768 fetch_fsmap learned FSMap version 29 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f516e548c0 con 0x55f516a69980 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6161600640 1 -- 172.21.15.27:0/2553612186 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6170011d50 con 0x55f516a69980 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6161600640 1 -- 172.21.15.27:0/2553612186 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6168044030 con 0x55f516a69980 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.547+0000 7f6185403740 10 client.25768 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.548+0000 7f6185403740 10 client.25768 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.548+0000 7f6185403740 20 client.25768 populate_metadata read hostname 'smithi027' 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.548+0000 7f6185403740 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a511c0 0x55f516e57e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.548+0000 7f6185403740 4 client.25768 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.548+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f516db1bf0 con 0x55f516a511c0 2024-09-21T01:25:17.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.549+0000 7f6176a00640 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a511c0 0x55f516e57e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.549+0000 7f6161600640 10 client.25768 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:17.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.549+0000 7f6176a00640 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a511c0 0x55f516e57e10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea0e1b16e02e921 server_cookie=2ad620d074b0dd02 in_seq=0 out_seq=0 2024-09-21T01:25:17.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6161600640 1 -- 172.21.15.27:0/2553612186 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f516db1bf0 con 0x55f516a511c0 2024-09-21T01:25:17.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6161600640 10 client.25768 handle_command_reply: tid=0 2024-09-21T01:25:17.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 client.25768 shutdown 2024-09-21T01:25:17.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 msgr2=0x55f516e45a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 0x55f516e45a90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f616802a910 tx=0x7f6168009260 comp rx=0 tx=0).stop 2024-09-21T01:25:17.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 shutdown_connections 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516e51e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a511c0 0x55f516e57e10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e45fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 --2- 172.21.15.27:0/2553612186 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a69980 0x55f516e45a90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 >> 172.21.15.27:0/2553612186 conn(0x55f516a86990 msgr2=0x55f516e369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6161600640 0 client.25768 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.551+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 shutdown_connections 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.552+0000 7f6185403740 1 -- 172.21.15.27:0/2553612186 wait complete. 2024-09-21T01:25:17.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.552+0000 7f6185403740 20 client.25768 trim_cache size 0 max 16384 2024-09-21T01:25:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.570+0000 7f6185403740 1 Processor -- start 2024-09-21T01:25:17.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.570+0000 7f6185403740 1 -- start start 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6185403740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6185403740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e44310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6185403740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a92840 0x55f516e47970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6185403740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61781b3130 con 0x55f516a69980 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6185403740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f61781b4c70 con 0x55f516a71b60 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6185403740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61781b3800 con 0x55f516a92840 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6177400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6176a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e44310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6176a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e44310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54892/0 (socket says 172.21.15.27:54892) 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6177400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45620/0 (socket says 172.21.15.27:45620) 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6177400640 1 -- 172.21.15.27:0/2785022481 learned_addr learned my addr 172.21.15.27:0/2785022481 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.571+0000 7f6177e00640 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a92840 0x55f516e47970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.572+0000 7f6176a00640 1 -- 172.21.15.27:0/2785022481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a92840 msgr2=0x55f516e47970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.572+0000 7f6176a00640 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a92840 0x55f516e47970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.572+0000 7f6176a00640 1 -- 172.21.15.27:0/2785022481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 msgr2=0x55f516e3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.572+0000 7f6176a00640 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.572+0000 7f6176a00640 1 -- 172.21.15.27:0/2785022481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f516e3c640 con 0x55f516a71b60 2024-09-21T01:25:17.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.572+0000 7f6176a00640 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e44310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61640e5210 tx=0x7f6164010e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f10113573b167f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.573+0000 7f6163e00640 1 -- 172.21.15.27:0/2785022481 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640a9040 con 0x55f516a71b60 2024-09-21T01:25:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.573+0000 7f6163e00640 1 -- 172.21.15.27:0/2785022481 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61640b23f0 con 0x55f516a71b60 2024-09-21T01:25:17.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.573+0000 7f6163e00640 1 -- 172.21.15.27:0/2785022481 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640ac5e0 con 0x55f516a71b60 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 -- 172.21.15.27:0/2785022481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 msgr2=0x55f516e44310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e44310 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61640e5210 tx=0x7f6164010e30 comp rx=0 tx=0).stop 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 -- 172.21.15.27:0/2785022481 shutdown_connections 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a92840 0x55f516e47970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a71b60 0x55f516e44310 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 --2- 172.21.15.27:0/2785022481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a69980 0x55f516e3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 -- 172.21.15.27:0/2785022481 >> 172.21.15.27:0/2785022481 conn(0x55f516a86990 msgr2=0x55f516e3d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 -- 172.21.15.27:0/2785022481 shutdown_connections 2024-09-21T01:25:17.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.574+0000 7f6185403740 1 -- 172.21.15.27:0/2785022481 wait complete. 2024-09-21T01:25:17.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.575+0000 7f6185403740 1 -- start start 2024-09-21T01:25:17.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.575+0000 7f6185403740 1 Processor -- start 2024-09-21T01:25:17.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 0x55f516e4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516ec1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61781b45a0 con 0x55f516a92840 2024-09-21T01:25:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f61781b3ed0 con 0x55f516a69980 2024-09-21T01:25:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61781b1110 con 0x55f516a71b60 2024-09-21T01:25:17.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6185403740 10 client.0 _ll_register_callbacks cb 0x7f6168009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6176a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 0x55f516e4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 0x55f516e4a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54908/0 (socket says 172.21.15.27:54908) 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 -- 172.21.15.27:0/3837936716 learned_addr learned my addr 172.21.15.27:0/3837936716 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 -- 172.21.15.27:0/3837936716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 msgr2=0x55f516e4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e4a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 -- 172.21.15.27:0/3837936716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 msgr2=0x55f516ec1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.576+0000 7f6177400640 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516ec1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.577+0000 7f6177400640 1 -- 172.21.15.27:0/3837936716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f516ec1d60 con 0x55f516a69980 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.577+0000 7f6177400640 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 0x55f516e4a110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f616803d7a0 tx=0x7f6168003900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e066293b28742 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.577+0000 7f6161600640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:25:17.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.577+0000 7f6161600640 1 -- 172.21.15.27:0/3837936716 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616803ddd0 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.577+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61640b23f0 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.577+0000 7f6161600640 1 -- 172.21.15.27:0/3837936716 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6168051470 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6161600640 1 -- 172.21.15.27:0/3837936716 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6168064680 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 10 client.25793 fetch_fsmap learned FSMap version 29 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f516ec2e30 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6161600640 1 -- 172.21.15.27:0/3837936716 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61640b23f0 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6161600640 1 -- 172.21.15.27:0/3837936716 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6168044040 con 0x55f516a69980 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 10 client.25793 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 10 client.25793 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 20 client.25793 populate_metadata read hostname 'smithi027' 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a26570 0x55f516ec5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 4 client.25793 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.578+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55f516dad6a0 con 0x55f516a26570 2024-09-21T01:25:17.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.579+0000 7f6176a00640 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a26570 0x55f516ec5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:17.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.579+0000 7f6176a00640 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a26570 0x55f516ec5620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737ee0534f7a580 server_cookie=93af242950241387 in_seq=0 out_seq=0 2024-09-21T01:25:17.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.579+0000 7f6161600640 10 client.25793 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:17.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6161600640 1 -- 172.21.15.27:0/3837936716 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f516dad6a0 con 0x55f516a26570 2024-09-21T01:25:17.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6161600640 10 client.25793 handle_command_reply: tid=0 2024-09-21T01:25:17.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 client.25793 shutdown 2024-09-21T01:25:17.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 msgr2=0x55f516e4a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 0x55f516e4a110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f616803d7a0 tx=0x7f6168003900 comp rx=0 tx=0).stop 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 shutdown_connections 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f516a92840 0x55f516ec1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f516a71b60 0x55f516e4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f516a26570 0x55f516ec5620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 --2- 172.21.15.27:0/3837936716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f516a69980 0x55f516e4a110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 >> 172.21.15.27:0/3837936716 conn(0x55f516a86990 msgr2=0x55f516e3de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.594+0000 7f6161600640 0 client.25793 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.595+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 shutdown_connections 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.595+0000 7f6185403740 1 -- 172.21.15.27:0/3837936716 wait complete. 2024-09-21T01:25:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.595+0000 7f6185403740 20 client.25793 trim_cache size 0 max 16384 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 14, 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "4c3d276f": { 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 14, 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:25:17.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:17.599 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f615c0b6680 msgr2=0x7f615c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f615c0b6680 0x7f615c0b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6168006010 tx=0x7f6168005a20 comp rx=0 tx=0).stop 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 msgr2=0x7f6178102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 0x7f6178102c50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f617806ab20 tx=0x7f616400ee50 comp rx=0 tx=0).stop 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 shutdown_connections 2024-09-21T01:25:17.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f615c0b6680 0x7f615c0b8b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617810de30 0x7f6178103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6178105320 0x7f6178102c50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 --2- 172.21.15.27:0/1514509751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61781049d0 0x7f6178102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:17.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.596+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 >> 172.21.15.27:0/1514509751 conn(0x7f6178079760 msgr2=0x7f6178100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:17.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.597+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 shutdown_connections 2024-09-21T01:25:17.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:17.597+0000 7f617e800640 1 -- 172.21.15.27:0/1514509751 wait complete. 2024-09-21T01:25:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:18 smithi098 ceph-mon[34372]: pgmap v486: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-21T01:25:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:25:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:18 smithi027 ceph-mon[30375]: pgmap v486: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-21T01:25:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:25:18.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:18 smithi089 ceph-mon[34486]: pgmap v486: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-09-21T01:25:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:25:18.543 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4c3d276f', quiesce took 4.5 seconds. Will release after: 23.3 2024-09-21T01:25:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:25:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:25:19.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:25:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:25:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:25:19.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:25:19.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:25:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:25:19.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:25:20.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:20 smithi098 ceph-mon[34372]: pgmap v487: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.2 KiB/s wr, 19 op/s 2024-09-21T01:25:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:20 smithi027 ceph-mon[30375]: pgmap v487: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.2 KiB/s wr, 19 op/s 2024-09-21T01:25:20.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:20 smithi089 ceph-mon[34486]: pgmap v487: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.2 KiB/s wr, 19 op/s 2024-09-21T01:25:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:22 smithi098 ceph-mon[34372]: pgmap v488: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 36 op/s 2024-09-21T01:25:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:22 smithi027 ceph-mon[30375]: pgmap v488: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 36 op/s 2024-09-21T01:25:22.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:22 smithi089 ceph-mon[34486]: pgmap v488: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 36 op/s 2024-09-21T01:25:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:24 smithi098 ceph-mon[34372]: pgmap v489: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 33 op/s 2024-09-21T01:25:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:24 smithi027 ceph-mon[30375]: pgmap v489: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 33 op/s 2024-09-21T01:25:24.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:24 smithi089 ceph-mon[34486]: pgmap v489: 129 pgs: 129 active+clean; 2.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 33 op/s 2024-09-21T01:25:26.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:26 smithi098 ceph-mon[34372]: pgmap v490: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-09-21T01:25:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:26 smithi027 ceph-mon[30375]: pgmap v490: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-09-21T01:25:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:26 smithi089 ceph-mon[34486]: pgmap v490: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.3 MiB/s wr, 34 op/s 2024-09-21T01:25:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:28 smithi027 ceph-mon[30375]: pgmap v491: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 37 op/s 2024-09-21T01:25:28.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:28 smithi089 ceph-mon[34486]: pgmap v491: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 37 op/s 2024-09-21T01:25:28.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:28 smithi098 ceph-mon[34372]: pgmap v491: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 37 op/s 2024-09-21T01:25:30.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:30 smithi027 ceph-mon[30375]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 35 op/s 2024-09-21T01:25:30.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:30 smithi089 ceph-mon[34486]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 35 op/s 2024-09-21T01:25:30.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:30 smithi098 ceph-mon[34372]: pgmap v492: 129 pgs: 129 active+clean; 2.2 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 35 op/s 2024-09-21T01:25:32.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:32 smithi027 ceph-mon[30375]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 29 op/s 2024-09-21T01:25:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:32 smithi089 ceph-mon[34486]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 29 op/s 2024-09-21T01:25:32.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:32 smithi098 ceph-mon[34372]: pgmap v493: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 29 op/s 2024-09-21T01:25:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:34 smithi027 ceph-mon[30375]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T01:25:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:34 smithi089 ceph-mon[34486]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T01:25:34.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:34 smithi098 ceph-mon[34372]: pgmap v494: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 20 MiB/s wr, 13 op/s 2024-09-21T01:25:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:36 smithi027 ceph-mon[30375]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-21T01:25:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:36 smithi089 ceph-mon[34486]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-21T01:25:36.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:36 smithi098 ceph-mon[34372]: pgmap v495: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 13 op/s 2024-09-21T01:25:37.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:37 smithi089 ceph-mon[34486]: pgmap v496: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-21T01:25:37.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:37 smithi098 ceph-mon[34372]: pgmap v496: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-21T01:25:37.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:37 smithi027 ceph-mon[30375]: pgmap v496: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-21T01:25:39.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:39 smithi098 ceph-mon[34372]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-21T01:25:39.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:39 smithi027 ceph-mon[30375]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-21T01:25:40.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:39 smithi089 ceph-mon[34486]: pgmap v497: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-21T01:25:41.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:25:41.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:41 smithi098 ceph-mon[34372]: pgmap v498: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-09-21T01:25:41.902 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:41 smithi027 ceph-mon[30375]: pgmap v498: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-09-21T01:25:41.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:41 smithi089 ceph-mon[34486]: pgmap v498: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.0 MiB/s wr, 13 op/s 2024-09-21T01:25:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:25:42.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 -- 172.21.15.27:0/2618990258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f277810b6c0 msgr2=0x7f277810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 --2- 172.21.15.27:0/2618990258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f277810b6c0 0x7f277810bad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2760009900 tx=0x7f276002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 -- 172.21.15.27:0/2618990258 shutdown_connections 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 --2- 172.21.15.27:0/2618990258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810c010 0x7f2778112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 --2- 172.21.15.27:0/2618990258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f277810b6c0 0x7f277810bad0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 --2- 172.21.15.27:0/2618990258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2778105d80 0x7f2778106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 -- 172.21.15.27:0/2618990258 >> 172.21.15.27:0/2618990258 conn(0x7f2778101810 msgr2=0x7f2778103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 -- 172.21.15.27:0/2618990258 shutdown_connections 2024-09-21T01:25:42.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.892+0000 7f277e000640 1 -- 172.21.15.27:0/2618990258 wait complete. 2024-09-21T01:25:42.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.893+0000 7f277e000640 1 Processor -- start 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 -- start start 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 0x7f277819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810b6c0 0x7f277819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f277810c010 0x7f27781a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2778115d20 con 0x7f2778105d80 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2778115ba0 con 0x7f277810b6c0 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f277e000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2778115ea0 con 0x7f277810c010 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f2777400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 0x7f277819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f276ea00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810b6c0 0x7f277819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f2777400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 0x7f277819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44552/0 (socket says 172.21.15.27:44552) 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.894+0000 7f2777400640 1 -- 172.21.15.27:0/3376994833 learned_addr learned my addr 172.21.15.27:0/3376994833 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:42.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f2777e00640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f277810c010 0x7f27781a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f276ea00640 1 -- 172.21.15.27:0/3376994833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f277810c010 msgr2=0x7f27781a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f276ea00640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f277810c010 0x7f27781a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f276ea00640 1 -- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 msgr2=0x7f277819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f276ea00640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 0x7f277819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f276ea00640 1 -- 172.21.15.27:0/3376994833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781a2660 con 0x7f277810b6c0 2024-09-21T01:25:42.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f2777400640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 0x7f277819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.895+0000 7f276ea00640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810b6c0 0x7f277819cc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2760009230 tx=0x7f2760003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.896+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276003f070 con 0x7f277810b6c0 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.896+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2760031b60 con 0x7f277810b6c0 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.896+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781a28f0 con 0x7f277810b6c0 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.896+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760035030 con 0x7f277810b6c0 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.896+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a90b0 con 0x7f277810b6c0 2024-09-21T01:25:42.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.897+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2778107000 con 0x7f277810b6c0 2024-09-21T01:25:42.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.900+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2760042070 con 0x7f277810b6c0 2024-09-21T01:25:42.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.902+0000 7f2774c00640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f275c0b6680 0x7f275c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:42.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.902+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f2760108840 con 0x7f277810b6c0 2024-09-21T01:25:42.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.902+0000 7f2777400640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f275c0b6680 0x7f275c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:42.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.903+0000 7f2777400640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f275c0b6680 0x7f275c0b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2768011500 tx=0x7f2768005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:42.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:42.903+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27600d2da0 con 0x7f277810b6c0 2024-09-21T01:25:43.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.143+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f277810dde0 con 0x7f277810b6c0 2024-09-21T01:25:43.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.145+0000 7f2774c00640 1 -- 172.21.15.27:0/3376994833 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f27600d7ce0 con 0x7f277810b6c0 2024-09-21T01:25:43.146 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:25:43.146 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:25:43.146 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:25:43.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.147+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f275c0b6680 msgr2=0x7f275c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:43.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.147+0000 7f277e000640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f275c0b6680 0x7f275c0b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2768011500 tx=0x7f2768005f90 comp rx=0 tx=0).stop 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.147+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810b6c0 msgr2=0x7f277819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.147+0000 7f277e000640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810b6c0 0x7f277819cc90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2760009230 tx=0x7f2760003c10 comp rx=0 tx=0).stop 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.147+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 shutdown_connections 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.147+0000 7f277e000640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f275c0b6680 0x7f275c0b8b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.148+0000 7f277e000640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f277810c010 0x7f27781a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.148+0000 7f277e000640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f277810b6c0 0x7f277819cc90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.148+0000 7f277e000640 1 --2- 172.21.15.27:0/3376994833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2778105d80 0x7f277819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.148+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 >> 172.21.15.27:0/3376994833 conn(0x7f2778101810 msgr2=0x7f2778103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.148+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 shutdown_connections 2024-09-21T01:25:43.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:43.148+0000 7f277e000640 1 -- 172.21.15.27:0/3376994833 wait complete. 2024-09-21T01:25:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:43 smithi098 ceph-mon[34372]: pgmap v499: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 MiB/s wr, 9 op/s 2024-09-21T01:25:43.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:43 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3376994833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:43.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:43 smithi027 ceph-mon[30375]: pgmap v499: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 MiB/s wr, 9 op/s 2024-09-21T01:25:43.930 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:43 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3376994833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:43.969 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db --set-id 4c3d276f --release --await' 2024-09-21T01:25:43.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db --set-id 4c3d276f --release --await 2024-09-21T01:25:43.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:43 smithi089 ceph-mon[34486]: pgmap v499: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 MiB/s wr, 9 op/s 2024-09-21T01:25:44.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:43 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3376994833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:25:44.356 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:25:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.087+0000 7fa05be00640 1 -- 172.21.15.27:0/3652334694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c109c40 msgr2=0x7fa05c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.087+0000 7fa05be00640 1 --2- 172.21.15.27:0/3652334694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c109c40 0x7fa05c110470 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa054009ca0 tx=0x7fa05402a8c0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.087+0000 7fa05be00640 1 -- 172.21.15.27:0/3652334694 shutdown_connections 2024-09-21T01:25:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.087+0000 7fa05be00640 1 --2- 172.21.15.27:0/3652334694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c109c40 0x7fa05c110470 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.088+0000 7fa05be00640 1 --2- 172.21.15.27:0/3652334694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c1092f0 0x7fa05c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.088+0000 7fa05be00640 1 --2- 172.21.15.27:0/3652334694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1039b0 0x7fa05c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.088+0000 7fa05be00640 1 -- 172.21.15.27:0/3652334694 >> 172.21.15.27:0/3652334694 conn(0x7fa05c0ff400 msgr2=0x7fa05c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.088+0000 7fa05be00640 1 -- 172.21.15.27:0/3652334694 shutdown_connections 2024-09-21T01:25:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.088+0000 7fa05be00640 1 -- 172.21.15.27:0/3652334694 wait complete. 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.089+0000 7fa05be00640 1 Processor -- start 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.089+0000 7fa05be00640 1 -- start start 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.089+0000 7fa05be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c1039b0 0x7fa05c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.089+0000 7fa05be00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 0x7fa05c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.089+0000 7fa05be00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c109c40 0x7fa05c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05be00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c113900 con 0x7fa05c1039b0 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05be00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c113780 con 0x7fa05c109c40 2024-09-21T01:25:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05be00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c113a80 con 0x7fa05c1092f0 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c1039b0 0x7fa05c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 0x7fa05c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 0x7fa05c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:35548/0 (socket says 172.21.15.27:35548) 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 -- 172.21.15.27:0/2580302121 learned_addr learned my addr 172.21.15.27:0/2580302121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05b400640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c109c40 0x7fa05c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 -- 172.21.15.27:0/2580302121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c109c40 msgr2=0x7fa05c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c109c40 0x7fa05c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 -- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c1039b0 msgr2=0x7fa05c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c1039b0 0x7fa05c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.090+0000 7fa05a000640 1 -- 172.21.15.27:0/2580302121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c1a0260 con 0x7fa05c1092f0 2024-09-21T01:25:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.091+0000 7fa05a000640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 0x7fa05c19a890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa04800e9f0 tx=0x7fa04800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.091+0000 7fa053400640 1 -- 172.21.15.27:0/2580302121 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04800ccc0 con 0x7fa05c1092f0 2024-09-21T01:25:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.091+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c1a0550 con 0x7fa05c1092f0 2024-09-21T01:25:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.091+0000 7fa053400640 1 -- 172.21.15.27:0/2580302121 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0480108e0 con 0x7fa05c1092f0 2024-09-21T01:25:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.091+0000 7fa053400640 1 -- 172.21.15.27:0/2580302121 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048018860 con 0x7fa05c1092f0 2024-09-21T01:25:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.091+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c1a6cb0 con 0x7fa05c1092f0 2024-09-21T01:25:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.094+0000 7fa068a21740 1 Processor -- start 2024-09-21T01:25:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 -- start start 2024-09-21T01:25:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe465340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe465880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe465dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5602fe3dc290 con 0x5602fe09d840 2024-09-21T01:25:45.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602fe3cb6c0 con 0x5602fe095660 2024-09-21T01:25:45.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa068a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5602fe3cf4d0 con 0x5602fe0be520 2024-09-21T01:25:45.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe465880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe465880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37416/0 (socket says 172.21.15.27:37416) 2024-09-21T01:25:45.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa05aa00640 1 -- 172.21.15.27:0/1748015972 learned_addr learned my addr 172.21.15.27:0/1748015972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:45.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.095+0000 7fa05b400640 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe465340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa05aa00640 1 -- 172.21.15.27:0/1748015972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 msgr2=0x5602fe465340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa05aa00640 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe465340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa05aa00640 1 -- 172.21.15.27:0/1748015972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 msgr2=0x5602fe465dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa05aa00640 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe465dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa05aa00640 1 -- 172.21.15.27:0/1748015972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602fe358460 con 0x5602fe095660 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa053400640 1 -- 172.21.15.27:0/2580302121 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fa048010430 con 0x7fa05c1092f0 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa05aa00640 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe465880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009d20 tx=0x7fa04c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e45c05931993024 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa047e00640 1 -- 172.21.15.27:0/1748015972 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c01e070 con 0x5602fe095660 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa047e00640 1 -- 172.21.15.27:0/1748015972 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c019ae0 con 0x5602fe095660 2024-09-21T01:25:45.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.096+0000 7fa047e00640 1 -- 172.21.15.27:0/1748015972 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00eab0 con 0x5602fe095660 2024-09-21T01:25:45.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 -- 172.21.15.27:0/1748015972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 msgr2=0x5602fe465880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe465880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009d20 tx=0x7fa04c011810 comp rx=0 tx=0).stop 2024-09-21T01:25:45.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 -- 172.21.15.27:0/1748015972 shutdown_connections 2024-09-21T01:25:45.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe465dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe465880 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 --2- 172.21.15.27:0/1748015972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe465340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 -- 172.21.15.27:0/1748015972 >> 172.21.15.27:0/1748015972 conn(0x5602fe0b2910 msgr2=0x5602fe463230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa053400640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa0400b6630 0x7fa0400b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.097+0000 7fa068a21740 1 -- 172.21.15.27:0/1748015972 shutdown_connections 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.098+0000 7fa068a21740 1 -- 172.21.15.27:0/1748015972 wait complete. 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.098+0000 7fa053400640 1 -- 172.21.15.27:0/2580302121 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa0480e7cc0 con 0x7fa05c1092f0 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.098+0000 7fa05aa00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa0400b6630 0x7fa0400b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.098+0000 7fa068a21740 1 -- start start 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 Processor -- start 2024-09-21T01:25:45.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa05aa00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa0400b6630 0x7fa0400b8b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e700 tx=0x7fa04c005850 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe475280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe4757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe4811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5602fe22cbb0 con 0x5602fe09d840 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602fe3666c0 con 0x5602fe095660 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5602fe477e50 con 0x5602fe0be520 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa068a21740 10 client.0 _ll_register_callbacks cb 0x5602fe381490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.099+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe475280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe475280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:37424/0 (socket says 172.21.15.27:37424) 2024-09-21T01:25:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05aa00640 1 -- 172.21.15.27:0/2606859475 learned_addr learned my addr 172.21.15.27:0/2606859475 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05a000640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe4757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe4811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 -- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 msgr2=0x5602fe475280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe475280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 -- 172.21.15.27:0/2606859475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 msgr2=0x5602fe4757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe4757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 -- 172.21.15.27:0/2606859475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602fe129b00 con 0x5602fe0be520 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05a000640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe4757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa05b400640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe4811b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa054006010 tx=0x7fa05402f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f718703eee8d788 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.100+0000 7fa044c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.101+0000 7fa044c00640 1 -- 172.21.15.27:0/2606859475 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054031660 con 0x5602fe0be520 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.101+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa04c019ae0 con 0x5602fe0be520 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.101+0000 7fa044c00640 1 -- 172.21.15.27:0/2606859475 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa054031800 con 0x5602fe0be520 2024-09-21T01:25:45.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.101+0000 7fa044c00640 1 -- 172.21.15.27:0/2606859475 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa054004b40 con 0x5602fe0be520 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.101+0000 7fa068a21740 10 client.25792 fetch_fsmap learned FSMap version 29 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.101+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602fe1a0960 con 0x5602fe0be520 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa044c00640 1 -- 172.21.15.27:0/2606859475 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c019ae0 con 0x5602fe0be520 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa044c00640 1 -- 172.21.15.27:0/2606859475 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa054004370 con 0x5602fe0be520 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa068a21740 10 client.25792 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa068a21740 10 client.25792 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa068a21740 20 client.25792 populate_metadata read hostname 'smithi027' 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa068a21740 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe07cea0 0x5602fe4871c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa068a21740 4 client.25792 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:25:45.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.102+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602fe3de1e0 con 0x5602fe07cea0 2024-09-21T01:25:45.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.103+0000 7fa05aa00640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe07cea0 0x5602fe4871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.103+0000 7fa05aa00640 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe07cea0 0x5602fe4871c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1aa50bee29985e server_cookie=47862d18d5dde28b in_seq=0 out_seq=0 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.103+0000 7fa044c00640 10 client.25792 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.104+0000 7fa044c00640 1 -- 172.21.15.27:0/2606859475 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602fe3de1e0 con 0x5602fe07cea0 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.104+0000 7fa044c00640 10 client.25792 handle_command_reply: tid=0 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.104+0000 7fa068a21740 1 client.25792 shutdown 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.104+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 msgr2=0x5602fe4811b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.104+0000 7fa068a21740 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe4811b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa054006010 tx=0x7fa05402f3a0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.104+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 shutdown_connections 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe0be520 0x5602fe4811b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe09d840 0x5602fe4757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe07cea0 0x5602fe4871c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 --2- 172.21.15.27:0/2606859475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe095660 0x5602fe475280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 >> 172.21.15.27:0/2606859475 conn(0x5602fe0b2910 msgr2=0x5602fe468490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa044c00640 0 client.25792 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:45.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 shutdown_connections 2024-09-21T01:25:45.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 1 -- 172.21.15.27:0/2606859475 wait complete. 2024-09-21T01:25:45.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.105+0000 7fa068a21740 20 client.25792 trim_cache size 0 max 16384 2024-09-21T01:25:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.123+0000 7fa068a21740 1 Processor -- start 2024-09-21T01:25:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.123+0000 7fa068a21740 1 -- start start 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.123+0000 7fa068a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe095660 0x5602fe4810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.123+0000 7fa068a21740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe09d840 0x5602fe4815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa068a21740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 0x5602fe4683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa068a21740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1aea00 con 0x5602fe0be520 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa068a21740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1b0540 con 0x5602fe09d840 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa068a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1af0d0 con 0x5602fe095660 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 0x5602fe4683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe095660 0x5602fe4810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05b400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 0x5602fe4683e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44576/0 (socket says 172.21.15.27:44576) 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05b400640 1 -- 172.21.15.27:0/3177647859 learned_addr learned my addr 172.21.15.27:0/3177647859 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05a000640 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe09d840 0x5602fe4815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05aa00640 1 -- 172.21.15.27:0/3177647859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe09d840 msgr2=0x5602fe4815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05aa00640 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe09d840 0x5602fe4815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05aa00640 1 -- 172.21.15.27:0/3177647859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 msgr2=0x5602fe4683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05aa00640 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 0x5602fe4683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05aa00640 1 -- 172.21.15.27:0/3177647859 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602fe45e1d0 con 0x5602fe095660 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05b400640 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 0x5602fe4683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:45.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.124+0000 7fa05a000640 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe09d840 0x5602fe4815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:25:45.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.125+0000 7fa05aa00640 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe095660 0x5602fe4810b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5602fe4878d0 tx=0x7fa04c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405016fd12eb64b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:45.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.125+0000 7fa047400640 1 -- 172.21.15.27:0/3177647859 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c01e040 con 0x5602fe095660 2024-09-21T01:25:45.127 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.125+0000 7fa047400640 1 -- 172.21.15.27:0/3177647859 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c005600 con 0x5602fe095660 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.125+0000 7fa047400640 1 -- 172.21.15.27:0/3177647859 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00edf0 con 0x5602fe095660 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 -- 172.21.15.27:0/3177647859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe095660 msgr2=0x5602fe4810b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe095660 0x5602fe4810b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5602fe4878d0 tx=0x7fa04c011810 comp rx=0 tx=0).stop 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 -- 172.21.15.27:0/3177647859 shutdown_connections 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe0be520 0x5602fe4683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe09d840 0x5602fe4815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 --2- 172.21.15.27:0/3177647859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe095660 0x5602fe4810b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 -- 172.21.15.27:0/3177647859 >> 172.21.15.27:0/3177647859 conn(0x5602fe0b2910 msgr2=0x5602fe473900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 -- 172.21.15.27:0/3177647859 shutdown_connections 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.126+0000 7fa068a21740 1 -- 172.21.15.27:0/3177647859 wait complete. 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.127+0000 7fa068a21740 1 -- start start 2024-09-21T01:25:45.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.127+0000 7fa068a21740 1 Processor -- start 2024-09-21T01:25:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.127+0000 7fa068a21740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe095660 0x5602fe479830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa068a21740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe09d840 0x5602fe479d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa068a21740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe0be520 0x5602fe4f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa068a21740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1afe70 con 0x5602fe095660 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa068a21740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1af7a0 con 0x5602fe0be520 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa068a21740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c1aca30 con 0x5602fe09d840 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa068a21740 10 client.0 _ll_register_callbacks cb 0x7fa054009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe095660 0x5602fe479830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05aa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe095660 0x5602fe479830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44578/0 (socket says 172.21.15.27:44578) 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05a000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe09d840 0x5602fe479d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05aa00640 1 -- 172.21.15.27:0/3792454606 learned_addr learned my addr 172.21.15.27:0/3792454606 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05b400640 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe0be520 0x5602fe4f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05a000640 1 -- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe0be520 msgr2=0x5602fe4f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05a000640 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe0be520 0x5602fe4f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.130 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05a000640 1 -- 172.21.15.27:0/3792454606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe095660 msgr2=0x5602fe479830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05a000640 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe095660 0x5602fe479830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.128+0000 7fa05a000640 1 -- 172.21.15.27:0/3792454606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602fe4f10a0 con 0x5602fe09d840 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa05a000640 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe09d840 0x5602fe479d70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0480e50a0 tx=0x7fa04800cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd3ebbc06e392a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa044c00640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa044c00640 1 -- 172.21.15.27:0/3792454606 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0480ad360 con 0x5602fe09d840 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa044c00640 1 -- 172.21.15.27:0/3792454606 <== mon.1 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa0480ac520 con 0x5602fe09d840 2024-09-21T01:25:45.131 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa04c005600 con 0x5602fe09d840 2024-09-21T01:25:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa044c00640 1 -- 172.21.15.27:0/3792454606 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0480b2280 con 0x5602fe09d840 2024-09-21T01:25:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa068a21740 10 client.25798 fetch_fsmap learned FSMap version 29 2024-09-21T01:25:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.129+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5602fe4f2370 con 0x5602fe09d840 2024-09-21T01:25:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa044c00640 1 -- 172.21.15.27:0/3792454606 <== mon.1 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c005600 con 0x5602fe09d840 2024-09-21T01:25:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa044c00640 1 -- 172.21.15.27:0/3792454606 <== mon.1 v2:172.21.15.98:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa0480b6700 con 0x5602fe09d840 2024-09-21T01:25:45.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa068a21740 10 client.25798 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa068a21740 10 client.25798 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa068a21740 20 client.25798 populate_metadata read hostname 'smithi027' 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa068a21740 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe052250 0x5602fe4f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa068a21740 4 client.25798 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4c3d276f", "release": true, "await": true}] 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4c3d276f", "release": true, "await": true}) -- 0x5602fe3d9c90 con 0x5602fe052250 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.130+0000 7fa05aa00640 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe052250 0x5602fe4f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.131+0000 7fa05aa00640 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe052250 0x5602fe4f4ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3613e687d47a9c server_cookie=ab06cdaf6b3affda in_seq=0 out_seq=0 2024-09-21T01:25:45.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.131+0000 7fa044c00640 10 client.25798 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa044c00640 1 -- 172.21.15.27:0/3792454606 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5602fe3d9c90 con 0x5602fe052250 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa044c00640 10 client.25798 handle_command_reply: tid=0 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 client.25798 shutdown 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe09d840 msgr2=0x5602fe479d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe09d840 0x5602fe479d70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0480e50a0 tx=0x7fa04800cef0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 shutdown_connections 2024-09-21T01:25:45.135 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x5602fe052250 0x5602fe4f4ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602fe0be520 0x5602fe4f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5602fe09d840 0x5602fe479d70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 --2- 172.21.15.27:0/3792454606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5602fe095660 0x5602fe479830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa044c00640 0 client.25798 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 >> 172.21.15.27:0/3792454606 conn(0x5602fe0b2910 msgr2=0x5602fe4683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 shutdown_connections 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.134+0000 7fa068a21740 1 -- 172.21.15.27:0/3792454606 wait complete. 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa068a21740 20 client.25798 trim_cache size 0 max 16384 2024-09-21T01:25:45.136 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 16, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "4c3d276f": { 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 16, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 147.8, 2024-09-21T01:25:45.137 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RELEASED", 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:25:45.138 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa0400b6630 msgr2=0x7fa0400b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa0400b6630 0x7fa0400b8b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e700 tx=0x7fa04c005850 comp rx=0 tx=0).stop 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 msgr2=0x7fa05c19a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 0x7fa05c19a890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa04800e9f0 tx=0x7fa04800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 shutdown_connections 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fa0400b6630 0x7fa0400b8b40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa05c109c40 0x7fa05c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa05c1092f0 0x7fa05c19a890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 --2- 172.21.15.27:0/2580302121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa05c1039b0 0x7fa05c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:25:45.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.135+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 >> 172.21.15.27:0/2580302121 conn(0x7fa05c0ff400 msgr2=0x7fa05c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:25:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.136+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 shutdown_connections 2024-09-21T01:25:45.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:25:45.136+0000 7fa05be00640 1 -- 172.21.15.27:0/2580302121 wait complete. 2024-09-21T01:25:45.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:45 smithi098 ceph-mon[34372]: pgmap v500: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:25:45.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:45 smithi027 ceph-mon[30375]: pgmap v500: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:25:45.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:45 smithi089 ceph-mon[34486]: pgmap v500: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:25:46.066 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4c3d276f', total seconds elapsed: 32.1 2024-09-21T01:25:47.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:47 smithi098 ceph-mon[34372]: pgmap v501: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:25:47.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:47 smithi027 ceph-mon[30375]: pgmap v501: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:25:47.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:47 smithi089 ceph-mon[34486]: pgmap v501: 129 pgs: 129 active+clean; 2.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:25:49.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:49 smithi098 ceph-mon[34372]: pgmap v502: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-21T01:25:49.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:49 smithi027 ceph-mon[30375]: pgmap v502: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-21T01:25:49.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:49 smithi089 ceph-mon[34486]: pgmap v502: 129 pgs: 129 active+clean; 2.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 MiB/s wr, 11 op/s 2024-09-21T01:25:51.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:51 smithi098 ceph-mon[34372]: pgmap v503: 129 pgs: 129 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 24 op/s 2024-09-21T01:25:51.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:51 smithi027 ceph-mon[30375]: pgmap v503: 129 pgs: 129 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 24 op/s 2024-09-21T01:25:51.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:51 smithi089 ceph-mon[34486]: pgmap v503: 129 pgs: 129 active+clean; 2.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 24 op/s 2024-09-21T01:25:53.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:53 smithi098 ceph-mon[34372]: pgmap v504: 129 pgs: 129 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 81 MiB/s wr, 30 op/s 2024-09-21T01:25:53.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:53 smithi027 ceph-mon[30375]: pgmap v504: 129 pgs: 129 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 81 MiB/s wr, 30 op/s 2024-09-21T01:25:53.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:53 smithi089 ceph-mon[34486]: pgmap v504: 129 pgs: 129 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 81 MiB/s wr, 30 op/s 2024-09-21T01:25:55.934 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:55 smithi027 ceph-mon[30375]: pgmap v505: 129 pgs: 129 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 115 MiB/s wr, 38 op/s 2024-09-21T01:25:55.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:55 smithi089 ceph-mon[34486]: pgmap v505: 129 pgs: 129 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 115 MiB/s wr, 38 op/s 2024-09-21T01:25:56.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:55 smithi098 ceph-mon[34372]: pgmap v505: 129 pgs: 129 active+clean; 3.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 115 MiB/s wr, 38 op/s 2024-09-21T01:25:57.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:57 smithi027 ceph-mon[30375]: pgmap v506: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 154 MiB/s wr, 52 op/s 2024-09-21T01:25:57.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:57 smithi089 ceph-mon[34486]: pgmap v506: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 154 MiB/s wr, 52 op/s 2024-09-21T01:25:58.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:57 smithi098 ceph-mon[34372]: pgmap v506: 129 pgs: 129 active+clean; 4.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 154 MiB/s wr, 52 op/s 2024-09-21T01:25:59.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:25:59 smithi027 ceph-mon[30375]: pgmap v507: 129 pgs: 129 active+clean; 4.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 210 MiB/s wr, 63 op/s 2024-09-21T01:25:59.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:25:59 smithi089 ceph-mon[34486]: pgmap v507: 129 pgs: 129 active+clean; 4.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 210 MiB/s wr, 63 op/s 2024-09-21T01:26:00.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:25:59 smithi098 ceph-mon[34372]: pgmap v507: 129 pgs: 129 active+clean; 4.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 210 MiB/s wr, 63 op/s 2024-09-21T01:26:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:01 smithi027 ceph-mon[30375]: pgmap v508: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-09-21T01:26:01.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:01 smithi089 ceph-mon[34486]: pgmap v508: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-09-21T01:26:02.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:01 smithi098 ceph-mon[34372]: pgmap v508: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 260 MiB/s wr, 80 op/s 2024-09-21T01:26:03.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:03 smithi089 ceph-mon[34486]: pgmap v509: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 266 MiB/s wr, 79 op/s 2024-09-21T01:26:04.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:03 smithi098 ceph-mon[34372]: pgmap v509: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 266 MiB/s wr, 79 op/s 2024-09-21T01:26:04.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:03 smithi027 ceph-mon[30375]: pgmap v509: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 266 MiB/s wr, 79 op/s 2024-09-21T01:26:05.984 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:05 smithi027 ceph-mon[30375]: pgmap v510: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 267 MiB/s wr, 93 op/s 2024-09-21T01:26:05.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:05 smithi089 ceph-mon[34486]: pgmap v510: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 267 MiB/s wr, 93 op/s 2024-09-21T01:26:06.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:05 smithi098 ceph-mon[34372]: pgmap v510: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 267 MiB/s wr, 93 op/s 2024-09-21T01:26:07.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:07 smithi089 ceph-mon[34486]: pgmap v511: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 271 MiB/s wr, 97 op/s 2024-09-21T01:26:08.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:07 smithi098 ceph-mon[34372]: pgmap v511: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 271 MiB/s wr, 97 op/s 2024-09-21T01:26:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:07 smithi027 ceph-mon[30375]: pgmap v511: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 271 MiB/s wr, 97 op/s 2024-09-21T01:26:09.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:09 smithi089 ceph-mon[34486]: pgmap v512: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 289 MiB/s wr, 98 op/s 2024-09-21T01:26:10.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:09 smithi098 ceph-mon[34372]: pgmap v512: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 289 MiB/s wr, 98 op/s 2024-09-21T01:26:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:09 smithi027 ceph-mon[30375]: pgmap v512: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 289 MiB/s wr, 98 op/s 2024-09-21T01:26:11.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:11 smithi089 ceph-mon[34486]: pgmap v513: 129 pgs: 129 active+clean; 8.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 292 MiB/s wr, 102 op/s 2024-09-21T01:26:12.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:11 smithi098 ceph-mon[34372]: pgmap v513: 129 pgs: 129 active+clean; 8.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 292 MiB/s wr, 102 op/s 2024-09-21T01:26:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:11 smithi027 ceph-mon[30375]: pgmap v513: 129 pgs: 129 active+clean; 8.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 292 MiB/s wr, 102 op/s 2024-09-21T01:26:13.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:13 smithi089 ceph-mon[34486]: pgmap v514: 129 pgs: 129 active+clean; 8.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 277 MiB/s wr, 94 op/s 2024-09-21T01:26:14.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:13 smithi098 ceph-mon[34372]: pgmap v514: 129 pgs: 129 active+clean; 8.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 277 MiB/s wr, 94 op/s 2024-09-21T01:26:14.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:13 smithi027 ceph-mon[30375]: pgmap v514: 129 pgs: 129 active+clean; 8.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 277 MiB/s wr, 94 op/s 2024-09-21T01:26:16.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:15 smithi089 ceph-mon[34486]: pgmap v515: 129 pgs: 129 active+clean; 9.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 282 MiB/s wr, 94 op/s 2024-09-21T01:26:16.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:15 smithi098 ceph-mon[34372]: pgmap v515: 129 pgs: 129 active+clean; 9.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 282 MiB/s wr, 94 op/s 2024-09-21T01:26:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:15 smithi027 ceph-mon[30375]: pgmap v515: 129 pgs: 129 active+clean; 9.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 282 MiB/s wr, 94 op/s 2024-09-21T01:26:17.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:17 smithi089 ceph-mon[34486]: pgmap v516: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 276 MiB/s wr, 82 op/s 2024-09-21T01:26:18.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:17 smithi098 ceph-mon[34372]: pgmap v516: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 276 MiB/s wr, 82 op/s 2024-09-21T01:26:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:17 smithi027 ceph-mon[30375]: pgmap v516: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 276 MiB/s wr, 82 op/s 2024-09-21T01:26:19.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:18 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:26:19.204 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:18 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:26:19.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:18 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:26:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:19 smithi098 ceph-mon[34372]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 294 MiB/s wr, 83 op/s 2024-09-21T01:26:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:26:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:26:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:19 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:26:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:19 smithi027 ceph-mon[30375]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 294 MiB/s wr, 83 op/s 2024-09-21T01:26:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:26:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:26:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:19 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:26:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:19 smithi089 ceph-mon[34486]: pgmap v517: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 294 MiB/s wr, 83 op/s 2024-09-21T01:26:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:26:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:26:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:19 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:26:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:21 smithi098 ceph-mon[34372]: pgmap v518: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 85 op/s 2024-09-21T01:26:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:21 smithi027 ceph-mon[30375]: pgmap v518: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 85 op/s 2024-09-21T01:26:22.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:21 smithi089 ceph-mon[34486]: pgmap v518: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 85 op/s 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 initial writers = 156519.17 kB/sec 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 initial writers = 156517.41 kB/sec 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 156519.17 kB/sec 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 156519.17 kB/sec 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 156519.17 kB/sec 2024-09-21T01:26:22.555 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-09-21T01:26:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:23 smithi098 ceph-mon[34372]: pgmap v519: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 268 MiB/s wr, 76 op/s 2024-09-21T01:26:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:23 smithi027 ceph-mon[30375]: pgmap v519: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 268 MiB/s wr, 76 op/s 2024-09-21T01:26:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:23 smithi089 ceph-mon[34486]: pgmap v519: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 268 MiB/s wr, 76 op/s 2024-09-21T01:26:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:25 smithi098 ceph-mon[34372]: pgmap v520: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 261 MiB/s wr, 75 op/s 2024-09-21T01:26:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:25 smithi027 ceph-mon[30375]: pgmap v520: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 261 MiB/s wr, 75 op/s 2024-09-21T01:26:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:25 smithi089 ceph-mon[34486]: pgmap v520: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 261 MiB/s wr, 75 op/s 2024-09-21T01:26:28.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:27 smithi098 ceph-mon[34372]: pgmap v521: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 236 MiB/s wr, 71 op/s 2024-09-21T01:26:28.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:27 smithi027 ceph-mon[30375]: pgmap v521: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 236 MiB/s wr, 71 op/s 2024-09-21T01:26:28.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:27 smithi089 ceph-mon[34486]: pgmap v521: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 236 MiB/s wr, 71 op/s 2024-09-21T01:26:30.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:29 smithi098 ceph-mon[34372]: pgmap v522: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 210 MiB/s wr, 61 op/s 2024-09-21T01:26:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:29 smithi027 ceph-mon[30375]: pgmap v522: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 210 MiB/s wr, 61 op/s 2024-09-21T01:26:30.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:29 smithi089 ceph-mon[34486]: pgmap v522: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 210 MiB/s wr, 61 op/s 2024-09-21T01:26:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:31 smithi098 ceph-mon[34372]: pgmap v523: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 56 op/s 2024-09-21T01:26:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:31 smithi027 ceph-mon[30375]: pgmap v523: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 56 op/s 2024-09-21T01:26:32.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:31 smithi089 ceph-mon[34486]: pgmap v523: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 56 op/s 2024-09-21T01:26:34.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:33 smithi098 ceph-mon[34372]: pgmap v524: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 146 MiB/s wr, 46 op/s 2024-09-21T01:26:34.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:33 smithi027 ceph-mon[30375]: pgmap v524: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 146 MiB/s wr, 46 op/s 2024-09-21T01:26:34.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:33 smithi089 ceph-mon[34486]: pgmap v524: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 146 MiB/s wr, 46 op/s 2024-09-21T01:26:36.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:35 smithi098 ceph-mon[34372]: pgmap v525: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 148 MiB/s wr, 47 op/s 2024-09-21T01:26:36.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:35 smithi027 ceph-mon[30375]: pgmap v525: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 148 MiB/s wr, 47 op/s 2024-09-21T01:26:36.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:35 smithi089 ceph-mon[34486]: pgmap v525: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 148 MiB/s wr, 47 op/s 2024-09-21T01:26:38.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:37 smithi098 ceph-mon[34372]: pgmap v526: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 144 MiB/s wr, 50 op/s 2024-09-21T01:26:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:37 smithi027 ceph-mon[30375]: pgmap v526: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 144 MiB/s wr, 50 op/s 2024-09-21T01:26:38.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:37 smithi089 ceph-mon[34486]: pgmap v526: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 144 MiB/s wr, 50 op/s 2024-09-21T01:26:40.184 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:39 smithi027 ceph-mon[30375]: pgmap v527: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 173 MiB/s wr, 54 op/s 2024-09-21T01:26:40.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:39 smithi089 ceph-mon[34486]: pgmap v527: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 173 MiB/s wr, 54 op/s 2024-09-21T01:26:40.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:39 smithi098 ceph-mon[34372]: pgmap v527: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 173 MiB/s wr, 54 op/s 2024-09-21T01:26:42.183 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:41 smithi027 ceph-mon[30375]: pgmap v528: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 215 MiB/s wr, 69 op/s 2024-09-21T01:26:42.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:41 smithi089 ceph-mon[34486]: pgmap v528: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 215 MiB/s wr, 69 op/s 2024-09-21T01:26:42.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:41 smithi098 ceph-mon[34372]: pgmap v528: 129 pgs: 129 active+clean; 15 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 215 MiB/s wr, 69 op/s 2024-09-21T01:26:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:43 smithi027 ceph-mon[30375]: pgmap v529: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 231 MiB/s wr, 69 op/s 2024-09-21T01:26:44.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:43 smithi089 ceph-mon[34486]: pgmap v529: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 231 MiB/s wr, 69 op/s 2024-09-21T01:26:44.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:43 smithi098 ceph-mon[34372]: pgmap v529: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 231 MiB/s wr, 69 op/s 2024-09-21T01:26:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:45 smithi027 ceph-mon[30375]: pgmap v530: 129 pgs: 129 active+clean; 15 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 239 MiB/s wr, 71 op/s 2024-09-21T01:26:46.199 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:45 smithi098 ceph-mon[34372]: pgmap v530: 129 pgs: 129 active+clean; 15 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 239 MiB/s wr, 71 op/s 2024-09-21T01:26:46.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:45 smithi089 ceph-mon[34486]: pgmap v530: 129 pgs: 129 active+clean; 15 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 239 MiB/s wr, 71 op/s 2024-09-21T01:26:48.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:47 smithi089 ceph-mon[34486]: pgmap v531: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 235 MiB/s wr, 74 op/s 2024-09-21T01:26:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:47 smithi098 ceph-mon[34372]: pgmap v531: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 235 MiB/s wr, 74 op/s 2024-09-21T01:26:48.435 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:47 smithi027 ceph-mon[30375]: pgmap v531: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 235 MiB/s wr, 74 op/s 2024-09-21T01:26:50.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:49 smithi089 ceph-mon[34486]: pgmap v532: 129 pgs: 129 active+clean; 16 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 257 MiB/s wr, 75 op/s 2024-09-21T01:26:50.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:49 smithi098 ceph-mon[34372]: pgmap v532: 129 pgs: 129 active+clean; 16 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 257 MiB/s wr, 75 op/s 2024-09-21T01:26:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:49 smithi027 ceph-mon[30375]: pgmap v532: 129 pgs: 129 active+clean; 16 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 257 MiB/s wr, 75 op/s 2024-09-21T01:26:52.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:51 smithi089 ceph-mon[34486]: pgmap v533: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 265 MiB/s wr, 82 op/s 2024-09-21T01:26:52.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:51 smithi098 ceph-mon[34372]: pgmap v533: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 265 MiB/s wr, 82 op/s 2024-09-21T01:26:52.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:51 smithi027 ceph-mon[30375]: pgmap v533: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 265 MiB/s wr, 82 op/s 2024-09-21T01:26:54.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:53 smithi089 ceph-mon[34486]: pgmap v534: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-09-21T01:26:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:53 smithi098 ceph-mon[34372]: pgmap v534: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-09-21T01:26:54.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:53 smithi027 ceph-mon[30375]: pgmap v534: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 253 MiB/s wr, 75 op/s 2024-09-21T01:26:56.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:55 smithi089 ceph-mon[34486]: pgmap v535: 129 pgs: 129 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 259 MiB/s wr, 76 op/s 2024-09-21T01:26:56.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:55 smithi098 ceph-mon[34372]: pgmap v535: 129 pgs: 129 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 259 MiB/s wr, 76 op/s 2024-09-21T01:26:56.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:55 smithi027 ceph-mon[30375]: pgmap v535: 129 pgs: 129 active+clean; 18 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 259 MiB/s wr, 76 op/s 2024-09-21T01:26:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:26:58 smithi098 ceph-mon[34372]: pgmap v536: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 257 MiB/s wr, 79 op/s 2024-09-21T01:26:58.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:26:57 smithi027 ceph-mon[30375]: pgmap v536: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 257 MiB/s wr, 79 op/s 2024-09-21T01:26:58.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:26:58 smithi089 ceph-mon[34486]: pgmap v536: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 257 MiB/s wr, 79 op/s 2024-09-21T01:27:00.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:00 smithi098 ceph-mon[34372]: pgmap v537: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 283 MiB/s wr, 81 op/s 2024-09-21T01:27:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:00 smithi027 ceph-mon[30375]: pgmap v537: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 283 MiB/s wr, 81 op/s 2024-09-21T01:27:00.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:00 smithi089 ceph-mon[34486]: pgmap v537: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 283 MiB/s wr, 81 op/s 2024-09-21T01:27:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:02 smithi098 ceph-mon[34372]: pgmap v538: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 282 MiB/s wr, 84 op/s 2024-09-21T01:27:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:02 smithi027 ceph-mon[30375]: pgmap v538: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 282 MiB/s wr, 84 op/s 2024-09-21T01:27:02.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:02 smithi089 ceph-mon[34486]: pgmap v538: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 282 MiB/s wr, 84 op/s 2024-09-21T01:27:04.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:04 smithi098 ceph-mon[34372]: pgmap v539: 129 pgs: 129 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 265 MiB/s wr, 77 op/s 2024-09-21T01:27:04.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:04 smithi027 ceph-mon[30375]: pgmap v539: 129 pgs: 129 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 265 MiB/s wr, 77 op/s 2024-09-21T01:27:04.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:04 smithi089 ceph-mon[34486]: pgmap v539: 129 pgs: 129 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 265 MiB/s wr, 77 op/s 2024-09-21T01:27:06.301 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:06 smithi089 ceph-mon[34486]: pgmap v540: 129 pgs: 129 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 261 MiB/s wr, 89 op/s 2024-09-21T01:27:06.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:06 smithi098 ceph-mon[34372]: pgmap v540: 129 pgs: 129 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 261 MiB/s wr, 89 op/s 2024-09-21T01:27:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:06 smithi027 ceph-mon[30375]: pgmap v540: 129 pgs: 129 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 261 MiB/s wr, 89 op/s 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 rewriters = 250169.27 kB/sec 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 rewriters = 250164.52 kB/sec 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 250169.27 kB/sec 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 250169.27 kB/sec 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 250169.27 kB/sec 2024-09-21T01:27:07.044 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-09-21T01:27:08.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:08 smithi098 ceph-mon[34372]: pgmap v541: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 255 MiB/s wr, 90 op/s 2024-09-21T01:27:08.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:08 smithi027 ceph-mon[30375]: pgmap v541: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 255 MiB/s wr, 90 op/s 2024-09-21T01:27:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:08 smithi089 ceph-mon[34486]: pgmap v541: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 255 MiB/s wr, 90 op/s 2024-09-21T01:27:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:10 smithi098 ceph-mon[34372]: pgmap v542: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 276 MiB/s wr, 92 op/s 2024-09-21T01:27:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:10 smithi027 ceph-mon[30375]: pgmap v542: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 276 MiB/s wr, 92 op/s 2024-09-21T01:27:10.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:10 smithi089 ceph-mon[34486]: pgmap v542: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 276 MiB/s wr, 92 op/s 2024-09-21T01:27:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:12 smithi098 ceph-mon[34372]: pgmap v543: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 89 op/s 2024-09-21T01:27:12.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:12 smithi027 ceph-mon[30375]: pgmap v543: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 89 op/s 2024-09-21T01:27:12.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:12 smithi089 ceph-mon[34486]: pgmap v543: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 249 MiB/s wr, 89 op/s 2024-09-21T01:27:12.636 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:27:12.636 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 readers = 5294822.00 kB/sec 2024-09-21T01:27:12.636 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 readers = 5292681.86 kB/sec 2024-09-21T01:27:12.637 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 5294822.00 kB/sec 2024-09-21T01:27:12.637 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 5294822.00 kB/sec 2024-09-21T01:27:12.637 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 5294822.00 kB/sec 2024-09-21T01:27:12.637 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-09-21T01:27:14.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:14 smithi098 ceph-mon[34372]: pgmap v544: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 195 MiB/s wr, 72 op/s 2024-09-21T01:27:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:14 smithi027 ceph-mon[30375]: pgmap v544: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 195 MiB/s wr, 72 op/s 2024-09-21T01:27:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:14 smithi089 ceph-mon[34486]: pgmap v544: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 195 MiB/s wr, 72 op/s 2024-09-21T01:27:16.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:16 smithi098 ceph-mon[34372]: pgmap v545: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 156 MiB/s wr, 61 op/s 2024-09-21T01:27:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:16 smithi027 ceph-mon[30375]: pgmap v545: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 156 MiB/s wr, 61 op/s 2024-09-21T01:27:16.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:16 smithi089 ceph-mon[34486]: pgmap v545: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 156 MiB/s wr, 61 op/s 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: Children see throughput for 1 re-readers = 3572169.25 kB/sec 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: Parent sees throughput for 1 re-readers = 3571221.34 kB/sec 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: Min throughput per process = 3572169.25 kB/sec 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: Max throughput per process = 3572169.25 kB/sec 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: Avg throughput per process = 3572169.25 kB/sec 2024-09-21T01:27:17.649 INFO:tasks.workunit.client.0.smithi027.stdout: Min xfer = 10485760.00 kB 2024-09-21T01:27:18.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:18 smithi098 ceph-mon[34372]: pgmap v546: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 124 MiB/s wr, 43 op/s 2024-09-21T01:27:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:18 smithi027 ceph-mon[30375]: pgmap v546: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 124 MiB/s wr, 43 op/s 2024-09-21T01:27:18.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:18 smithi089 ceph-mon[34486]: pgmap v546: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 124 MiB/s wr, 43 op/s 2024-09-21T01:27:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:20 smithi098 ceph-mon[34372]: pgmap v547: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 90 MiB/s wr, 31 op/s 2024-09-21T01:27:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:20 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:27:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:20 smithi027 ceph-mon[30375]: pgmap v547: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 90 MiB/s wr, 31 op/s 2024-09-21T01:27:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:20 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:27:20.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:20 smithi089 ceph-mon[34486]: pgmap v547: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 90 MiB/s wr, 31 op/s 2024-09-21T01:27:20.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:20 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:27:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:27:21.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:27:21.399 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:27:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:27:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:27:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:22 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T01:27:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:22 smithi098 ceph-mon[34372]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T01:27:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:22 smithi098 ceph-mon[34372]: pgmap v548: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 34 MiB/s wr, 20 op/s 2024-09-21T01:27:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:22 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T01:27:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:22 smithi027 ceph-mon[30375]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T01:27:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:22 smithi027 ceph-mon[30375]: pgmap v548: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 34 MiB/s wr, 20 op/s 2024-09-21T01:27:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:22 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi027 to 176.4M 2024-09-21T01:27:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:22 smithi089 ceph-mon[34486]: Unable to set osd_memory_target on smithi027 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-21T01:27:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:22 smithi089 ceph-mon[34486]: pgmap v548: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 34 MiB/s wr, 20 op/s 2024-09-21T01:27:24.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:24 smithi098 ceph-mon[34372]: pgmap v549: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.0 MiB/s wr, 9 op/s 2024-09-21T01:27:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:24 smithi027 ceph-mon[30375]: pgmap v549: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.0 MiB/s wr, 9 op/s 2024-09-21T01:27:24.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:24 smithi089 ceph-mon[34486]: pgmap v549: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.0 MiB/s wr, 9 op/s 2024-09-21T01:27:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:26 smithi098 ceph-mon[34372]: pgmap v550: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:27:26.432 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:26 smithi027 ceph-mon[30375]: pgmap v550: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:27:26.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:26 smithi089 ceph-mon[34486]: pgmap v550: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-21T01:27:28.072 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:27:28.072 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:27:28.080 INFO:tasks.workunit.client.0.smithi027.stdout: 2024-09-21T01:27:28.080 INFO:tasks.workunit.client.0.smithi027.stdout:iozone test complete. 2024-09-21T01:27:28.084 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T01:27:28.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T01:27:28.218 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-21T01:27:28.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T01:27:28.314 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:28 smithi027 ceph-mon[30375]: pgmap v551: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-21T01:27:28.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:28 smithi098 ceph-mon[34372]: pgmap v551: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-21T01:27:28.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:28 smithi089 ceph-mon[34486]: pgmap v551: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-21T01:27:29.325 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T01:27:29.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:27:29.712 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:27:29.742 DEBUG:teuthology.parallel:result is None 2024-09-21T01:27:29.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T01:27:29.809 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T01:27:29.809 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T01:27:29.868 INFO:tasks.quiescer:joining Quiescers 2024-09-21T01:27:29.868 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T01:27:29.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:27:30.310 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:30 smithi027 ceph-mon[30375]: pgmap v552: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 74 op/s 2024-09-21T01:27:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 -- 172.21.15.27:0/3113542013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b40760a0 msgr2=0x7f90b410fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 --2- 172.21.15.27:0/3113542013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b40760a0 0x7f90b410fbe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009860 tx=0x7f90a002a340 comp rx=0 tx=0).stop 2024-09-21T01:27:30.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 -- 172.21.15.27:0/3113542013 shutdown_connections 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 --2- 172.21.15.27:0/3113542013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4110120 0x7f90b4112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 --2- 172.21.15.27:0/3113542013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b40760a0 0x7f90b410fbe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 --2- 172.21.15.27:0/3113542013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b4075750 0x7f90b4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 -- 172.21.15.27:0/3113542013 >> 172.21.15.27:0/3113542013 conn(0x7f90b407b850 msgr2=0x7f90b407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 -- 172.21.15.27:0/3113542013 shutdown_connections 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.391+0000 7f90b8e00640 1 -- 172.21.15.27:0/3113542013 wait complete. 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 Processor -- start 2024-09-21T01:27:30.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 -- start start 2024-09-21T01:27:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 0x7f90b419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40760a0 0x7f90b419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b4110120 0x7f90b41a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:30.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4115ca0 con 0x7f90b40760a0 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4115b20 con 0x7f90b4075750 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.393+0000 7f90b8e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4115e20 con 0x7f90b4110120 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40760a0 0x7f90b419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 0x7f90b419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 0x7f90b419c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38544/0 (socket says 172.21.15.27:38544) 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b2800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b4110120 0x7f90b41a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 -- 172.21.15.27:0/1216672972 learned_addr learned my addr 172.21.15.27:0/1216672972 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:30.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40760a0 0x7f90b419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50614/0 (socket says 172.21.15.27:50614) 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 -- 172.21.15.27:0/1216672972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b4110120 msgr2=0x7f90b41a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b4110120 0x7f90b41a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 -- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40760a0 msgr2=0x7f90b419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40760a0 0x7f90b419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 -- 172.21.15.27:0/1216672972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b41a26c0 con 0x7f90b4075750 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.394+0000 7f90b1e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 0x7f90b419c750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9f0 tx=0x7f90a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.395+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400cd80 con 0x7f90b4075750 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.395+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b41a29b0 con 0x7f90b4075750 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.395+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a40108e0 con 0x7f90b4075750 2024-09-21T01:27:30.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.395+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4018980 con 0x7f90b4075750 2024-09-21T01:27:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.395+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41a9110 con 0x7f90b4075750 2024-09-21T01:27:30.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.396+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940052c0 con 0x7f90b4075750 2024-09-21T01:27:30.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:30 smithi098 ceph-mon[34372]: pgmap v552: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 74 op/s 2024-09-21T01:27:30.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.399+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f90a4010430 con 0x7f90b4075750 2024-09-21T01:27:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.400+0000 7f90aaa00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90980b66b0 0x7f90980b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.401+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f90a40e7e30 con 0x7f90b4075750 2024-09-21T01:27:30.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.401+0000 7f90b1400640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90980b66b0 0x7f90980b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:30.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.401+0000 7f90b1400640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90980b66b0 0x7f90980b8bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90a002a8b0 tx=0x7f90a002a840 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:30.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.402+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90a40b2390 con 0x7f90b4075750 2024-09-21T01:27:30.438 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:27:30.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:30 smithi089 ceph-mon[34486]: pgmap v552: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 74 op/s 2024-09-21T01:27:30.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.644+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90940055b0 con 0x7f90b4075750 2024-09-21T01:27:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.645+0000 7f90aaa00640 1 -- 172.21.15.27:0/1216672972 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7f90a40b72d0 con 0x7f90b4075750 2024-09-21T01:27:30.647 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:27:30.647 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:27:30.647 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:27:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.647+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90980b66b0 msgr2=0x7f90980b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.647+0000 7f90b8e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90980b66b0 0x7f90980b8bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90a002a8b0 tx=0x7f90a002a840 comp rx=0 tx=0).stop 2024-09-21T01:27:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.647+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 msgr2=0x7f90b419c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.647+0000 7f90b8e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 0x7f90b419c750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9f0 tx=0x7f90a400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 shutdown_connections 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f90980b66b0 0x7f90980b8bc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90b4110120 0x7f90b41a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90b40760a0 0x7f90b419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 --2- 172.21.15.27:0/1216672972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90b4075750 0x7f90b419c750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 >> 172.21.15.27:0/1216672972 conn(0x7f90b407b850 msgr2=0x7f90b4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 shutdown_connections 2024-09-21T01:27:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:30.648+0000 7f90b8e00640 1 -- 172.21.15.27:0/1216672972 wait complete. 2024-09-21T01:27:31.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:31 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1216672972' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:27:31.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:31 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1216672972' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:27:31.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:31 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1216672972' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:27:31.617 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T01:27:31.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 -- 172.21.15.27:0/3714714163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410c020 msgr2=0x7fdbb4112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/3714714163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410c020 0x7fdbb4112850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02a410 comp rx=0 tx=0).stop 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 -- 172.21.15.27:0/3714714163 shutdown_connections 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/3714714163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410c020 0x7fdbb4112850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/3714714163 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb410b6d0 0x7fdbb410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/3714714163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4105d90 0x7fdbb41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 -- 172.21.15.27:0/3714714163 >> 172.21.15.27:0/3714714163 conn(0x7fdbb4101860 msgr2=0x7fdbb4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:31.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 -- 172.21.15.27:0/3714714163 shutdown_connections 2024-09-21T01:27:31.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.944+0000 7fdbbaa00640 1 -- 172.21.15.27:0/3714714163 wait complete. 2024-09-21T01:27:31.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 Processor -- start 2024-09-21T01:27:31.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 -- start start 2024-09-21T01:27:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb4105d90 0x7fdbb410fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410b6d0 0x7fdbb41101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 0x7fdbb4110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4115ce0 con 0x7fdbb410b6d0 2024-09-21T01:27:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4115b60 con 0x7fdbb410c020 2024-09-21T01:27:31.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.946+0000 7fdbbaa00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4115e60 con 0x7fdbb4105d90 2024-09-21T01:27:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 0x7fdbb4110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410b6d0 0x7fdbb41101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb3e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb4105d90 0x7fdbb410fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbab400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410b6d0 0x7fdbb41101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50634/0 (socket says 172.21.15.27:50634) 2024-09-21T01:27:31.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbab400640 1 -- 172.21.15.27:0/4190225078 learned_addr learned my addr 172.21.15.27:0/4190225078 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb8a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 0x7fdbb4110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38562/0 (socket says 172.21.15.27:38562) 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb3e00640 1 -- 172.21.15.27:0/4190225078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 msgr2=0x7fdbb4110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb3e00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 0x7fdbb4110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb3e00640 1 -- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410b6d0 msgr2=0x7fdbb41101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb3e00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410b6d0 0x7fdbb41101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb3e00640 1 -- 172.21.15.27:0/4190225078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb41acb50 con 0x7fdbb4105d90 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.947+0000 7fdbb8a00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 0x7fdbb4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:27:31.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.948+0000 7fdbb3e00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb4105d90 0x7fdbb410fc90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdba400eb00 tx=0x7fdba400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.948+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4010430 con 0x7fdbb4105d90 2024-09-21T01:27:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.948+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba4010a80 con 0x7fdbb4105d90 2024-09-21T01:27:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.948+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4017a90 con 0x7fdbb4105d90 2024-09-21T01:27:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.948+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb41ace40 con 0x7fdbb4105d90 2024-09-21T01:27:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.948+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb41ad240 con 0x7fdbb4105d90 2024-09-21T01:27:31.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.950+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb41061a0 con 0x7fdbb4105d90 2024-09-21T01:27:31.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.952+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fdba4017c70 con 0x7fdbb4105d90 2024-09-21T01:27:31.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.954+0000 7fdbb1600640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdb980b6680 0x7fdb980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:31.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.954+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdba40e8820 con 0x7fdbb4105d90 2024-09-21T01:27:31.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.954+0000 7fdbab400640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdb980b6680 0x7fdb980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:31.955 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.955+0000 7fdbab400640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdb980b6680 0x7fdb980b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdbb41050b0 tx=0x7fdb9c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:31.956 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:31.956+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdba40ae9a0 con 0x7fdbb4105d90 2024-09-21T01:27:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:27:32.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.203+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb4078f10 con 0x7fdbb4105d90 2024-09-21T01:27:32.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.204+0000 7fdbb1600640 1 -- 172.21.15.27:0/4190225078 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fdba40b2d80 con 0x7fdbb4105d90 2024-09-21T01:27:32.205 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:27:32.205 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:27:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:27:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdb980b6680 msgr2=0x7fdb980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdb980b6680 0x7fdb980b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdbb41050b0 tx=0x7fdb9c005f90 comp rx=0 tx=0).stop 2024-09-21T01:27:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb4105d90 msgr2=0x7fdbb410fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb4105d90 0x7fdbb410fc90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdba400eb00 tx=0x7fdba400c2d0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 shutdown_connections 2024-09-21T01:27:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fdb980b6680 0x7fdb980b8b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb410c020 0x7fdbb4110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb410b6d0 0x7fdbb41101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 --2- 172.21.15.27:0/4190225078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbb4105d90 0x7fdbb410fc90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 >> 172.21.15.27:0/4190225078 conn(0x7fdbb4101860 msgr2=0x7fdbb410ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 shutdown_connections 2024-09-21T01:27:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.207+0000 7fdbbaa00640 1 -- 172.21.15.27:0/4190225078 wait complete. 2024-09-21T01:27:32.334 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:32 smithi027 ceph-mon[30375]: pgmap v553: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 184 op/s 2024-09-21T01:27:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:32 smithi098 ceph-mon[34372]: pgmap v553: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 184 op/s 2024-09-21T01:27:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:32 smithi089 ceph-mon[34486]: pgmap v553: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 184 op/s 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.810+0000 7f638fc00640 1 -- 172.21.15.27:0/727053438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 msgr2=0x7f638810fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.810+0000 7f638fc00640 1 --2- 172.21.15.27:0/727053438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638810fba0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6374009860 tx=0x7f637402a340 comp rx=0 tx=0).stop 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.810+0000 7f638fc00640 1 -- 172.21.15.27:0/727053438 shutdown_connections 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.811+0000 7f638fc00640 1 --2- 172.21.15.27:0/727053438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63881100e0 0x7f6388112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.811+0000 7f638fc00640 1 --2- 172.21.15.27:0/727053438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638810fba0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.811+0000 7f638fc00640 1 --2- 172.21.15.27:0/727053438 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63880757b0 0x7f6388075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.811+0000 7f638fc00640 1 -- 172.21.15.27:0/727053438 >> 172.21.15.27:0/727053438 conn(0x7f63880ff5d0 msgr2=0x7f6388101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.811+0000 7f638fc00640 1 -- 172.21.15.27:0/727053438 shutdown_connections 2024-09-21T01:27:32.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.811+0000 7f638fc00640 1 -- 172.21.15.27:0/727053438 wait complete. 2024-09-21T01:27:32.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.812+0000 7f638fc00640 1 Processor -- start 2024-09-21T01:27:32.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.812+0000 7f638fc00640 1 -- start start 2024-09-21T01:27:32.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.812+0000 7f638fc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 0x7f638819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638fc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638fc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881100e0 0x7f63881a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638fc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6388115c90 con 0x7f63880757b0 2024-09-21T01:27:32.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638fc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6388115b10 con 0x7f63881100e0 2024-09-21T01:27:32.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638fc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6388115e10 con 0x7f6388076100 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 0x7f638819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638d200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 0x7f638819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50662/0 (socket says 172.21.15.27:50662) 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638d200640 1 -- 172.21.15.27:0/1587773564 learned_addr learned my addr 172.21.15.27:0/1587773564 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f6387e00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.813+0000 7f638dc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881100e0 0x7f63881a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6387e00640 1 -- 172.21.15.27:0/1587773564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881100e0 msgr2=0x7f63881a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6387e00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881100e0 0x7f63881a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6387e00640 1 -- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 msgr2=0x7f638819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6387e00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 0x7f638819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6387e00640 1 -- 172.21.15.27:0/1587773564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63881a2610 con 0x7f6388076100 2024-09-21T01:27:32.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f638d200640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 0x7f638819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:27:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6387e00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638819cc40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6374035040 tx=0x7f6374032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f6385600640 1 -- 172.21.15.27:0/1587773564 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374040020 con 0x7f6388076100 2024-09-21T01:27:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.814+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63881a28a0 con 0x7f6388076100 2024-09-21T01:27:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.815+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63881a9060 con 0x7f6388076100 2024-09-21T01:27:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.815+0000 7f6385600640 1 -- 172.21.15.27:0/1587773564 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6374030e10 con 0x7f6388076100 2024-09-21T01:27:32.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.815+0000 7f6385600640 1 -- 172.21.15.27:0/1587773564 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374037040 con 0x7f6388076100 2024-09-21T01:27:32.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.818+0000 7f63968d0740 1 Processor -- start 2024-09-21T01:27:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.818+0000 7f63968d0740 1 -- start start 2024-09-21T01:27:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f63968d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957443350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f63968d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957443890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f63968d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957443dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f63968d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5579573b8ca0 con 0x55795707ab60 2024-09-21T01:27:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f63968d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5579573a80d0 con 0x557957072980 2024-09-21T01:27:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f63968d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5579573abee0 con 0x55795709b840 2024-09-21T01:27:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f638dc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957443350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f638dc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957443350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:44844/0 (socket says 172.21.15.27:44844) 2024-09-21T01:27:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f638dc00640 1 -- 172.21.15.27:0/1047011785 learned_addr learned my addr 172.21.15.27:0/1047011785 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:32.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.819+0000 7f638d200640 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957443890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f638d200640 1 -- 172.21.15.27:0/1047011785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 msgr2=0x557957443350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f638d200640 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957443350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f638d200640 1 -- 172.21.15.27:0/1047011785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 msgr2=0x557957443dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f638d200640 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957443dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f638d200640 1 -- 172.21.15.27:0/1047011785 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795735a400 con 0x557957072980 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f638dc00640 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957443350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.820+0000 7f6385600640 1 -- 172.21.15.27:0/1587773564 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6374043020 con 0x7f6388076100 2024-09-21T01:27:32.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.821+0000 7f638d200640 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957443890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f637c011510 tx=0x7f637c0119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b77ec116e77e172 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.821+0000 7f637a000640 1 -- 172.21.15.27:0/1047011785 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c01e070 con 0x557957072980 2024-09-21T01:27:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.821+0000 7f637a000640 1 -- 172.21.15.27:0/1047011785 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637c00fa70 con 0x557957072980 2024-09-21T01:27:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.821+0000 7f637a000640 1 -- 172.21.15.27:0/1047011785 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c019b50 con 0x557957072980 2024-09-21T01:27:32.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.822+0000 7f6385600640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f63700b6700 0x7f63700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.822+0000 7f6385600640 1 -- 172.21.15.27:0/1587773564 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6374107260 con 0x7f6388076100 2024-09-21T01:27:32.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.822+0000 7f638dc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f63700b6700 0x7f63700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.822+0000 7f63968d0740 1 -- 172.21.15.27:0/1047011785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 msgr2=0x557957443890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.822+0000 7f63968d0740 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957443890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f637c011510 tx=0x7f637c0119d0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.823+0000 7f63968d0740 1 -- 172.21.15.27:0/1047011785 shutdown_connections 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.823+0000 7f63968d0740 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957443dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.823+0000 7f63968d0740 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957443890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.823+0000 7f63968d0740 1 --2- 172.21.15.27:0/1047011785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957443350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.823+0000 7f63968d0740 1 -- 172.21.15.27:0/1047011785 >> 172.21.15.27:0/1047011785 conn(0x55795708f990 msgr2=0x557957441240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.823+0000 7f63968d0740 1 -- 172.21.15.27:0/1047011785 shutdown_connections 2024-09-21T01:27:32.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.824+0000 7f63968d0740 1 -- 172.21.15.27:0/1047011785 wait complete. 2024-09-21T01:27:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.824+0000 7f638dc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f63700b6700 0x7f63700b8c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6380009d60 tx=0x7f6380009500 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.824+0000 7f63968d0740 1 -- start start 2024-09-21T01:27:32.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.824+0000 7f63968d0740 1 Processor -- start 2024-09-21T01:27:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957457220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957457760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957463620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5579572097a0 con 0x55795707ab60 2024-09-21T01:27:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55795734da10 con 0x557957072980 2024-09-21T01:27:32.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55795745a260 con 0x55795709b840 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.825+0000 7f63968d0740 10 client.0 _ll_register_callbacks cb 0x55795735dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957457220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f6387e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957457760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957457220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38592/0 (socket says 172.21.15.27:38592) 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 -- 172.21.15.27:0/1502523106 learned_addr learned my addr 172.21.15.27:0/1502523106 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638dc00640 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957463620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 -- 172.21.15.27:0/1502523106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 msgr2=0x557957463620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957463620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 -- 172.21.15.27:0/1502523106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 msgr2=0x557957457760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957457760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.826+0000 7f638d200640 1 -- 172.21.15.27:0/1502523106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557957115b90 con 0x557957072980 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f638d200640 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957457220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f637c009880 tx=0x7f637c009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bd7c230d4d8701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:32.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f636ea00640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-09-21T01:27:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f636ea00640 1 -- 172.21.15.27:0/1502523106 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c026030 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637c00fa70 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f636ea00640 1 -- 172.21.15.27:0/1502523106 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f637c016070 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f636ea00640 1 -- 172.21.15.27:0/1502523106 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f637c01e040 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f63968d0740 10 client.25841 fetch_fsmap learned FSMap version 29 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.827+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557957466040 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.828+0000 7f636ea00640 1 -- 172.21.15.27:0/1502523106 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f637c00fa70 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.828+0000 7f636ea00640 1 -- 172.21.15.27:0/1502523106 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f637c021030 con 0x557957072980 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.828+0000 7f63968d0740 10 client.25841 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.828+0000 7f63968d0740 10 client.25841 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.829+0000 7f63968d0740 20 client.25841 populate_metadata read hostname 'smithi027' 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.829+0000 7f63968d0740 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795705a1c0 0x557957469400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.829+0000 7f63968d0740 4 client.25841 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:27:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.829+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579573babf0 con 0x55795705a1c0 2024-09-21T01:27:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.829+0000 7f6387e00640 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795705a1c0 0x557957469400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.830+0000 7f6387e00640 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795705a1c0 0x557957469400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2668ffbd55384e54 server_cookie=6783fc18f6295ca7 in_seq=0 out_seq=0 2024-09-21T01:27:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.830+0000 7f636ea00640 10 client.25841 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:32.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f636ea00640 1 -- 172.21.15.27:0/1502523106 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5579573babf0 con 0x55795705a1c0 2024-09-21T01:27:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f636ea00640 10 client.25841 handle_command_reply: tid=0 2024-09-21T01:27:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 client.25841 shutdown 2024-09-21T01:27:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 msgr2=0x557957457220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957457220 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f637c009880 tx=0x7f637c009de0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 shutdown_connections 2024-09-21T01:27:32.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x557957463620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x557957457760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795705a1c0 0x557957469400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 --2- 172.21.15.27:0/1502523106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957457220 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 >> 172.21.15.27:0/1502523106 conn(0x55795708f990 msgr2=0x55795743f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.831+0000 7f636ea00640 0 client.25841 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.832+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 shutdown_connections 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.832+0000 7f63968d0740 1 -- 172.21.15.27:0/1502523106 wait complete. 2024-09-21T01:27:32.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.832+0000 7f63968d0740 20 client.25841 trim_cache size 0 max 16384 2024-09-21T01:27:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 Processor -- start 2024-09-21T01:27:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 -- start start 2024-09-21T01:27:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957458740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795707ab60 0x557957463520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795709b840 0x557957454410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f636800fec0 con 0x55795709b840 2024-09-21T01:27:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6368011a00 con 0x557957072980 2024-09-21T01:27:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.852+0000 7f63968d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6368010590 con 0x55795707ab60 2024-09-21T01:27:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f638d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957458740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f638d200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957458740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38596/0 (socket says 172.21.15.27:38596) 2024-09-21T01:27:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f638d200640 1 -- 172.21.15.27:0/650587736 learned_addr learned my addr 172.21.15.27:0/650587736 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f6387e00640 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795707ab60 0x557957463520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f638dc00640 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795709b840 0x557957454410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f6387e00640 1 -- 172.21.15.27:0/650587736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 msgr2=0x557957458740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f6387e00640 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957458740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f6387e00640 1 -- 172.21.15.27:0/650587736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795709b840 msgr2=0x557957454410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f6387e00640 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795709b840 0x557957454410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f6387e00640 1 -- 172.21.15.27:0/650587736 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557957459dd0 con 0x55795707ab60 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.853+0000 7f638d200640 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957458740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.854+0000 7f6387e00640 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795707ab60 0x557957463520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f63741045e0 tx=0x7f6374005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aac9ef0cb3d0abe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.854+0000 7f6379600640 1 -- 172.21.15.27:0/650587736 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63740c8b80 con 0x55795707ab60 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.854+0000 7f6379600640 1 -- 172.21.15.27:0/650587736 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63740cc7f0 con 0x55795707ab60 2024-09-21T01:27:32.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.854+0000 7f6379600640 1 -- 172.21.15.27:0/650587736 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63740c90c0 con 0x55795707ab60 2024-09-21T01:27:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.855+0000 7f63968d0740 1 -- 172.21.15.27:0/650587736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795707ab60 msgr2=0x557957463520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.855+0000 7f63968d0740 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795707ab60 0x557957463520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f63741045e0 tx=0x7f6374005c90 comp rx=0 tx=0).stop 2024-09-21T01:27:32.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 -- 172.21.15.27:0/650587736 shutdown_connections 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795709b840 0x557957454410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795707ab60 0x557957463520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 --2- 172.21.15.27:0/650587736 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x557957458740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 -- 172.21.15.27:0/650587736 >> 172.21.15.27:0/650587736 conn(0x55795708f990 msgr2=0x557957444d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 -- 172.21.15.27:0/650587736 shutdown_connections 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.856+0000 7f63968d0740 1 -- 172.21.15.27:0/650587736 wait complete. 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.857+0000 7f63968d0740 1 -- start start 2024-09-21T01:27:32.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.857+0000 7f63968d0740 1 Processor -- start 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x55795745bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x55795745c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x5579574d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6368011330 con 0x55795707ab60 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6368010c60 con 0x557957072980 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f636800dea0 con 0x55795709b840 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f63968d0740 10 client.0 _ll_register_callbacks cb 0x7f637c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f6387e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x55795745c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f6387e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x55795745c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50672/0 (socket says 172.21.15.27:50672) 2024-09-21T01:27:32.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f6387e00640 1 -- 172.21.15.27:0/687976457 learned_addr learned my addr 172.21.15.27:0/687976457 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f638d200640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x55795745bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.858+0000 7f638dc00640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x5579574d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f6387e00640 1 -- 172.21.15.27:0/687976457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 msgr2=0x5579574d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f6387e00640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x5579574d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f6387e00640 1 -- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 msgr2=0x55795745bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f6387e00640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x55795745bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f6387e00640 1 -- 172.21.15.27:0/687976457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579574d31f0 con 0x55795707ab60 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f6387e00640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x55795745c140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6374104640 tx=0x7f6374033fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc8ddc3fb8cbc04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f636ea00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f636ea00640 1 -- 172.21.15.27:0/687976457 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63740b8020 con 0x55795707ab60 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f636ea00640 1 -- 172.21.15.27:0/687976457 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f63740f00b0 con 0x55795707ab60 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.859+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63740cc7f0 con 0x55795707ab60 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.860+0000 7f636ea00640 1 -- 172.21.15.27:0/687976457 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63740cd3e0 con 0x55795707ab60 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.860+0000 7f63968d0740 10 client.15792 fetch_fsmap learned FSMap version 29 2024-09-21T01:27:32.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.860+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579574d43b0 con 0x55795707ab60 2024-09-21T01:27:32.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.860+0000 7f636ea00640 1 -- 172.21.15.27:0/687976457 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63740cc7f0 con 0x55795707ab60 2024-09-21T01:27:32.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f636ea00640 1 -- 172.21.15.27:0/687976457 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6374030450 con 0x55795707ab60 2024-09-21T01:27:32.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f63968d0740 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:27:32.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f63968d0740 10 client.15792 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f63968d0740 20 client.15792 populate_metadata read hostname 'smithi027' 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f63968d0740 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795702f570 0x5579574d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f63968d0740 4 client.15792 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x5579573b66a0 con 0x55795702f570 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.861+0000 7f638d200640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795702f570 0x5579574d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.862+0000 7f638d200640 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795702f570 0x5579574d6af0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a8ec4e51c344cc server_cookie=822334dba23beafb in_seq=0 out_seq=0 2024-09-21T01:27:32.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:32.862+0000 7f636ea00640 10 client.15792 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:33.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:33 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4190225078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:27:33.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:33 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4190225078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:27:33.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:33 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4190225078' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:27:33.545 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24776 quiesce db --cancel --all' 2024-09-21T01:27:33.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24776 quiesce db --cancel --all 2024-09-21T01:27:33.934 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:27:34.117 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:34 smithi027 ceph-mon[30375]: pgmap v554: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 215 op/s 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f636ea00640 1 -- 172.21.15.27:0/687976457 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5579573b66a0 con 0x55795702f570 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f636ea00640 10 client.15792 handle_command_reply: tid=0 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 client.15792 shutdown 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 msgr2=0x55795745c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x55795745c140 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6374104640 tx=0x7f6374033fc0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 shutdown_connections 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55795709b840 0x5579574d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55795707ab60 0x55795745c140 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55795702f570 0x5579574d6af0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 --2- 172.21.15.27:0/687976457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557957072980 0x55795745bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f636ea00640 0 client.15792 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.241+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 >> 172.21.15.27:0/687976457 conn(0x55795708f990 msgr2=0x557957444d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.242+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 shutdown_connections 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.242+0000 7f63968d0740 1 -- 172.21.15.27:0/687976457 wait complete. 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.242+0000 7f63968d0740 20 client.15792 trim_cache size 0 max 16384 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 18, 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": { 2024-09-21T01:27:34.244 INFO:teuthology.orchestra.run.smithi027.stdout: "e672ba80": { 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "version": 18, 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "age_ref": 0.0, 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "timeout": 90.0, 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "expiration": 130.0, 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "members": { 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "file:/": { 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "excluded": false, 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "state": { 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "QUIESCED", 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: "age": 0.0 2024-09-21T01:27:34.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:27:34.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:27:34.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:27:34.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:27:34.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:27:34.246 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.242+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f63700b6700 msgr2=0x7f63700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.242+0000 7f638fc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f63700b6700 0x7f63700b8c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6380009d60 tx=0x7f6380009500 comp rx=0 tx=0).stop 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 msgr2=0x7f638819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638819cc40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6374035040 tx=0x7f6374032d50 comp rx=0 tx=0).stop 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 shutdown_connections 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f63700b6700 0x7f63700b8c10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63881100e0 0x7f63881a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6388076100 0x7f638819cc40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 --2- 172.21.15.27:0/1587773564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63880757b0 0x7f638819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 >> 172.21.15.27:0/1587773564 conn(0x7f63880ff5d0 msgr2=0x7f6388109c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 shutdown_connections 2024-09-21T01:27:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.243+0000 7f638fc00640 1 -- 172.21.15.27:0/1587773564 wait complete. 2024-09-21T01:27:34.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:34 smithi098 ceph-mon[34372]: pgmap v554: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 215 op/s 2024-09-21T01:27:34.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:34 smithi089 ceph-mon[34486]: pgmap v554: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 215 op/s 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.804+0000 7f6bad800640 1 -- 172.21.15.27:0/1239412082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 msgr2=0x7f6ba81061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.804+0000 7f6bad800640 1 --2- 172.21.15.27:0/1239412082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba81061a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009860 tx=0x7f6b9802a340 comp rx=0 tx=0).stop 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 -- 172.21.15.27:0/1239412082 shutdown_connections 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 --2- 172.21.15.27:0/1239412082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba810c020 0x7f6ba8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 --2- 172.21.15.27:0/1239412082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba810b6d0 0x7f6ba810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 --2- 172.21.15.27:0/1239412082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba81061a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 -- 172.21.15.27:0/1239412082 >> 172.21.15.27:0/1239412082 conn(0x7f6ba8101860 msgr2=0x7f6ba8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 -- 172.21.15.27:0/1239412082 shutdown_connections 2024-09-21T01:27:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.805+0000 7f6bad800640 1 -- 172.21.15.27:0/1239412082 wait complete. 2024-09-21T01:27:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.806+0000 7f6bad800640 1 Processor -- start 2024-09-21T01:27:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.806+0000 7f6bad800640 1 -- start start 2024-09-21T01:27:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.806+0000 7f6bad800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6bad800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba810b6d0 0x7f6ba819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6bad800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba810c020 0x7f6ba81a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6bad800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8115ce0 con 0x7f6ba810c020 2024-09-21T01:27:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6bad800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8115b60 con 0x7f6ba8105d90 2024-09-21T01:27:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6bad800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8115e60 con 0x7f6ba810b6d0 2024-09-21T01:27:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38616/0 (socket says 172.21.15.27:38616) 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 -- 172.21.15.27:0/87652342 learned_addr learned my addr 172.21.15.27:0/87652342 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba7400640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba810c020 0x7f6ba81a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba810b6d0 0x7f6ba819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 -- 172.21.15.27:0/87652342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba810b6d0 msgr2=0x7f6ba819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba810b6d0 0x7f6ba819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 -- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba810c020 msgr2=0x7f6ba81a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba810c020 0x7f6ba81a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.807+0000 7f6ba6a00640 1 -- 172.21.15.27:0/87652342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba81a2640 con 0x7f6ba8105d90 2024-09-21T01:27:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.808+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba819c6d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b98035040 tx=0x7f6b98032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.808+0000 7f6b9f400640 1 -- 172.21.15.27:0/87652342 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98037040 con 0x7f6ba8105d90 2024-09-21T01:27:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.808+0000 7f6b9f400640 1 -- 172.21.15.27:0/87652342 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b98041070 con 0x7f6ba8105d90 2024-09-21T01:27:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.808+0000 7f6b9f400640 1 -- 172.21.15.27:0/87652342 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980035f0 con 0x7f6ba8105d90 2024-09-21T01:27:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.808+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba81a28d0 con 0x7f6ba8105d90 2024-09-21T01:27:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.809+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba81a2d00 con 0x7f6ba8105d90 2024-09-21T01:27:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.812+0000 7f6bb4418740 1 Processor -- start 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.812+0000 7f6bb4418740 1 -- start start 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.812+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77aba02d0 0x55f77af46290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.812+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af46d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6bb4418740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f77aebdff0 con 0x55f77ab77410 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6bb4418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f77aead420 con 0x55f77ab7f5f0 2024-09-21T01:27:34.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6bb4418740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f77aeb1230 con 0x55f77aba02d0 2024-09-21T01:27:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af46d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6ba7400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77aba02d0 0x55f77af46290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af46d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:38626/0 (socket says 172.21.15.27:38626) 2024-09-21T01:27:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3993879079 learned_addr learned my addr 172.21.15.27:0/3993879079 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3993879079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77aba02d0 msgr2=0x55f77af46290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9f400640 1 -- 172.21.15.27:0/87652342 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6b98004050 con 0x7f6ba8105d90 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77aba02d0 0x55f77af46290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3993879079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 msgr2=0x55f77af467d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af467d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3993879079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77ac0b8b0 con 0x55f77ab7f5f0 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.813+0000 7f6ba7400640 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77aba02d0 0x55f77af46290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:27:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.814+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af46d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009c20 tx=0x7f6b90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a4f15ec932d152 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.814+0000 7f6b97e00640 1 -- 172.21.15.27:0/3993879079 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90004230 con 0x55f77ab7f5f0 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.814+0000 7f6b97e00640 1 -- 172.21.15.27:0/3993879079 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b900043d0 con 0x55f77ab7f5f0 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.814+0000 7f6b97e00640 1 -- 172.21.15.27:0/3993879079 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90004ca0 con 0x55f77ab7f5f0 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 -- 172.21.15.27:0/3993879079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 msgr2=0x55f77af46d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af46d10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009c20 tx=0x7f6b90011810 comp rx=0 tx=0).stop 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 -- 172.21.15.27:0/3993879079 shutdown_connections 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af46d10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3993879079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77aba02d0 0x55f77af46290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 -- 172.21.15.27:0/3993879079 >> 172.21.15.27:0/3993879079 conn(0x55f77ab946c0 msgr2=0x55f77af44180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 -- 172.21.15.27:0/3993879079 shutdown_connections 2024-09-21T01:27:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6bb4418740 1 -- 172.21.15.27:0/3993879079 wait complete. 2024-09-21T01:27:34.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.815+0000 7f6b9f400640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6b8c0b6610 0x7f6b8c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.816+0000 7f6b9f400640 1 -- 172.21.15.27:0/87652342 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6b98107730 con 0x7f6ba8105d90 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.816+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6b8c0b6610 0x7f6b8c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.816+0000 7f6bb4418740 1 -- start start 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.816+0000 7f6bb4418740 1 Processor -- start 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6b8c0b6610 0x7f6b8c0b8b20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6b900047e0 tx=0x7f6b90005850 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af51940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab7f5f0 0x55f77af51e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77aba02d0 0x55f77af523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f77ad0e810 con 0x55f77ab77410 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f77ae48550 con 0x55f77aba02d0 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f77af54a70 con 0x55f77ab7f5f0 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6bb4418740 10 client.0 _ll_register_callbacks cb 0x55f77ae58040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af51940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af51940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50702/0 (socket says 172.21.15.27:50702) 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6ba6a00640 1 -- 172.21.15.27:0/2209721159 learned_addr learned my addr 172.21.15.27:0/2209721159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab7f5f0 0x55f77af51e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.817+0000 7f6ba7400640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77aba02d0 0x55f77af523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba6a00640 1 -- 172.21.15.27:0/2209721159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab7f5f0 msgr2=0x55f77af51e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab7f5f0 0x55f77af51e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba6a00640 1 -- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77aba02d0 msgr2=0x55f77af523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77aba02d0 0x55f77af523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba6a00640 1 -- 172.21.15.27:0/2209721159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77ae3a200 con 0x55f77ab77410 2024-09-21T01:27:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba7400640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77aba02d0 0x55f77af523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:27:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af51940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b980cceb0 tx=0x7f6b980cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4390dfde2a709c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6b94c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:27:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6b94c00640 1 -- 172.21.15.27:0/2209721159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980328f0 con 0x55f77ab77410 2024-09-21T01:27:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6b94c00640 1 -- 172.21.15.27:0/2209721159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6b980cc410 con 0x55f77ab77410 2024-09-21T01:27:34.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.818+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b900043d0 con 0x55f77ab77410 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6b94c00640 1 -- 172.21.15.27:0/2209721159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b980cccc0 con 0x55f77ab77410 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f77af60600 con 0x55f77ab77410 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6b94c00640 1 -- 172.21.15.27:0/2209721159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b900043d0 con 0x55f77ab77410 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6b94c00640 1 -- 172.21.15.27:0/2209721159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6b980b81d0 con 0x55f77ab77410 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 10 client.15801 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 20 client.15801 populate_metadata read hostname 'smithi027' 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77ab5ec50 0x55f77af63a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 4 client.15801 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T01:27:34.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.819+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f77aebff40 con 0x55f77ab5ec50 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.820+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77ab5ec50 0x55f77af63a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.820+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77ab5ec50 0x55f77af63a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28aabef1213c46a server_cookie=c51bcaba32c92270 in_seq=0 out_seq=0 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.820+0000 7f6b94c00640 10 client.15801 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6b94c00640 1 -- 172.21.15.27:0/2209721159 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f77aebff40 con 0x55f77ab5ec50 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6b94c00640 10 client.15801 handle_command_reply: tid=0 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 client.15801 shutdown 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 msgr2=0x55f77af51940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af51940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b980cceb0 tx=0x7f6b980cd040 comp rx=0 tx=0).stop 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 shutdown_connections 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77ab5ec50 0x55f77af63a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77aba02d0 0x55f77af523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab7f5f0 0x55f77af51e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 --2- 172.21.15.27:0/2209721159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77ab77410 0x55f77af51940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 >> 172.21.15.27:0/2209721159 conn(0x55f77ab946c0 msgr2=0x55f77af49420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.821+0000 7f6b94c00640 0 client.15801 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.822+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 shutdown_connections 2024-09-21T01:27:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.822+0000 7f6bb4418740 1 -- 172.21.15.27:0/2209721159 wait complete. 2024-09-21T01:27:34.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.822+0000 7f6bb4418740 20 client.15801 trim_cache size 0 max 16384 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 Processor -- start 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 -- start start 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab77410 0x55f77af43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af5d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b3060 con 0x55f77aba02d0 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b4ba0 con 0x55f77ab7f5f0 2024-09-21T01:27:34.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6bb4418740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b3730 con 0x55f77ab77410 2024-09-21T01:27:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6ba7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af5d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6ba6a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab77410 0x55f77af43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6ba7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af5d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50704/0 (socket says 172.21.15.27:50704) 2024-09-21T01:27:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6ba7400640 1 -- 172.21.15.27:0/1661605410 learned_addr learned my addr 172.21.15.27:0/1661605410 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:34.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.841+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6ba6a00640 1 -- 172.21.15.27:0/1661605410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 msgr2=0x55f77af53760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af53760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6ba6a00640 1 -- 172.21.15.27:0/1661605410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 msgr2=0x55f77af5d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af5d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6ba6a00640 1 -- 172.21.15.27:0/1661605410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77af48580 con 0x55f77ab77410 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab77410 0x55f77af43180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b980c9780 tx=0x7f6b980cd0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5def63a2a75e428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6b97400640 1 -- 172.21.15.27:0/1661605410 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980328f0 con 0x55f77ab77410 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6b97400640 1 -- 172.21.15.27:0/1661605410 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b980cd8b0 con 0x55f77ab77410 2024-09-21T01:27:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6b97400640 1 -- 172.21.15.27:0/1661605410 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98107340 con 0x55f77ab77410 2024-09-21T01:27:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6bb4418740 1 -- 172.21.15.27:0/1661605410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab77410 msgr2=0x55f77af43180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.842+0000 7f6bb4418740 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab77410 0x55f77af43180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b980c9780 tx=0x7f6b980cd0b0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 -- 172.21.15.27:0/1661605410 shutdown_connections 2024-09-21T01:27:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af5d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 --2- 172.21.15.27:0/1661605410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab77410 0x55f77af43180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 -- 172.21.15.27:0/1661605410 >> 172.21.15.27:0/1661605410 conn(0x55f77ab946c0 msgr2=0x55f77af5cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 -- 172.21.15.27:0/1661605410 shutdown_connections 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 -- 172.21.15.27:0/1661605410 wait complete. 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 -- start start 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.844+0000 7f6bb4418740 1 Processor -- start 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af5a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af56880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab34000 0x55f77af56dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b44d0 con 0x55f77aba02d0 2024-09-21T01:27:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b3e00 con 0x55f77ab7f5f0 2024-09-21T01:27:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba81b1040 con 0x55f77ab34000 2024-09-21T01:27:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 10 client.0 _ll_register_callbacks cb 0x7f6b980c9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T01:27:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af56880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af56880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50720/0 (socket says 172.21.15.27:50720) 2024-09-21T01:27:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3592417256 learned_addr learned my addr 172.21.15.27:0/3592417256 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af5a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6ba7400640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab34000 0x55f77af56dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3592417256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab34000 msgr2=0x55f77af56dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab34000 0x55f77af56dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 msgr2=0x55f77af5a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af5a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 -- 172.21.15.27:0/3592417256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77af57430 con 0x55f77aba02d0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b9fe00640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af56880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004810 tx=0x7f6b900048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958d291bf6d99b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b94c00640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b94c00640 1 -- 172.21.15.27:0/3592417256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9001e050 con 0x55f77aba02d0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b94c00640 1 -- 172.21.15.27:0/3592417256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6b90026980 con 0x55f77aba02d0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b980cd8b0 con 0x55f77aba02d0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b94c00640 1 -- 172.21.15.27:0/3592417256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b90026b80 con 0x55f77aba02d0 2024-09-21T01:27:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 10 client.15807 fetch_fsmap learned FSMap version 29 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f77afcf110 con 0x55f77aba02d0 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b94c00640 1 -- 172.21.15.27:0/3592417256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b980cd8b0 con 0x55f77aba02d0 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6b94c00640 1 -- 172.21.15.27:0/3592417256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6b90018070 con 0x55f77aba02d0 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 10 client.15807 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 10 client.15807 resolve_mds: validated gid 24776 aka daemon mds.b 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 20 client.15807 populate_metadata read hostname 'smithi027' 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.845+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77afcf6a0 0x55f77afd1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 4 client.15807 mds_command: new command op to 24776 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 --> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f77aebb9f0 con 0x55f77afcf6a0 2024-09-21T01:27:34.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77afcf6a0 0x55f77afd1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:27:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6ba6a00640 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77afcf6a0 0x55f77afd1ac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778a6c02e34a84bb server_cookie=a43fdfd9aa05b70a in_seq=0 out_seq=0 2024-09-21T01:27:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6b94c00640 10 client.15807 ms_handle_connect on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6b94c00640 1 -- 172.21.15.27:0/3592417256 <== mds.0 v2:172.21.15.89:6838/4156411896 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f77aebb9f0 con 0x55f77afcf6a0 2024-09-21T01:27:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6b94c00640 10 client.15807 handle_command_reply: tid=0 2024-09-21T01:27:34.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 client.15807 shutdown 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 msgr2=0x55f77af56880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af56880 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b90004810 tx=0x7f6b900048e0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 shutdown_connections 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f77ab34000 0x55f77af56dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f77aba02d0 0x55f77af56880 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:6838/4156411896,v1:172.21.15.89:6839/4156411896] conn(0x55f77afcf6a0 0x55f77afd1ac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 --2- 172.21.15.27:0/3592417256 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f77ab7f5f0 0x55f77af5a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6b94c00640 0 client.15807 ms_handle_reset on v2:172.21.15.89:6838/4156411896 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 >> 172.21.15.27:0/3592417256 conn(0x55f77ab946c0 msgr2=0x55f77af5d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 shutdown_connections 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 1 -- 172.21.15.27:0/3592417256 wait complete. 2024-09-21T01:27:34.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bb4418740 20 client.15807 trim_cache size 0 max 16384 2024-09-21T01:27:34.853 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:27:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 29, 2024-09-21T01:27:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "leader": 24776, 2024-09-21T01:27:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "set_version": 19, 2024-09-21T01:27:34.853 INFO:teuthology.orchestra.run.smithi027.stdout: "sets": {} 2024-09-21T01:27:34.853 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6b8c0b6610 msgr2=0x7f6b8c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6b8c0b6610 0x7f6b8c0b8b20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6b900047e0 tx=0x7f6b90005850 comp rx=0 tx=0).stop 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 msgr2=0x7f6ba819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba819c6d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b98035040 tx=0x7f6b98032d50 comp rx=0 tx=0).stop 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 shutdown_connections 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6b8c0b6610 0x7f6b8c0b8b20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba810c020 0x7f6ba81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba810b6d0 0x7f6ba819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 --2- 172.21.15.27:0/87652342 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ba8105d90 0x7f6ba819c6d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 >> 172.21.15.27:0/87652342 conn(0x7f6ba8101860 msgr2=0x7f6ba8103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 shutdown_connections 2024-09-21T01:27:34.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:27:34.848+0000 7f6bad800640 1 -- 172.21.15.27:0/87652342 wait complete. 2024-09-21T01:27:35.399 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e672ba80', quiesce took 6.1 seconds. Will release after: 3.9000000000000004 2024-09-21T01:27:36.198 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T01:27:36.198 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T01:27:36.208 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T01:27:36.208 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T01:27:36.208 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:27:36.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T01:27:36.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T01:27:36.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:36 smithi098 ceph-mon[34372]: pgmap v555: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 223 op/s 2024-09-21T01:27:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:36 smithi027 ceph-mon[30375]: pgmap v555: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 223 op/s 2024-09-21T01:27:36.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:36 smithi089 ceph-mon[34486]: pgmap v555: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 223 op/s 2024-09-21T01:27:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:38 smithi098 ceph-mon[34372]: pgmap v556: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 KiB/s wr, 226 op/s 2024-09-21T01:27:38.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:38 smithi027 ceph-mon[30375]: pgmap v556: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 KiB/s wr, 226 op/s 2024-09-21T01:27:38.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:38 smithi089 ceph-mon[34486]: pgmap v556: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 KiB/s wr, 226 op/s 2024-09-21T01:27:40.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:40 smithi098 ceph-mon[34372]: pgmap v557: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 KiB/s wr, 222 op/s 2024-09-21T01:27:40.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:40 smithi027 ceph-mon[30375]: pgmap v557: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 KiB/s wr, 222 op/s 2024-09-21T01:27:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:40 smithi089 ceph-mon[34486]: pgmap v557: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 KiB/s wr, 222 op/s 2024-09-21T01:27:42.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:42 smithi098 ceph-mon[34372]: pgmap v558: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 161 op/s 2024-09-21T01:27:42.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:42 smithi027 ceph-mon[30375]: pgmap v558: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 161 op/s 2024-09-21T01:27:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:42 smithi089 ceph-mon[34486]: pgmap v558: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 KiB/s wr, 161 op/s 2024-09-21T01:27:44.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:44 smithi098 ceph-mon[34372]: pgmap v559: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 51 op/s 2024-09-21T01:27:44.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:44 smithi027 ceph-mon[30375]: pgmap v559: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 51 op/s 2024-09-21T01:27:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:44 smithi089 ceph-mon[34486]: pgmap v559: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 51 op/s 2024-09-21T01:27:46.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:46 smithi098 ceph-mon[34372]: pgmap v560: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 18 op/s 2024-09-21T01:27:46.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:46 smithi027 ceph-mon[30375]: pgmap v560: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 18 op/s 2024-09-21T01:27:46.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:46 smithi089 ceph-mon[34486]: pgmap v560: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 18 op/s 2024-09-21T01:27:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:48 smithi098 ceph-mon[34372]: pgmap v561: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-21T01:27:48.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:48 smithi027 ceph-mon[30375]: pgmap v561: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-21T01:27:48.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:48 smithi089 ceph-mon[34486]: pgmap v561: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-21T01:27:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:50 smithi098 ceph-mon[34372]: pgmap v562: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:27:50.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:50 smithi027 ceph-mon[30375]: pgmap v562: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:27:50.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:50 smithi089 ceph-mon[34486]: pgmap v562: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T01:27:52.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:52 smithi098 ceph-mon[34372]: pgmap v563: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:27:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:52 smithi027 ceph-mon[30375]: pgmap v563: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:27:52.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:52 smithi089 ceph-mon[34486]: pgmap v563: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:27:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:54 smithi098 ceph-mon[34372]: pgmap v564: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:27:54.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:54 smithi027 ceph-mon[30375]: pgmap v564: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:27:54.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:54 smithi089 ceph-mon[34486]: pgmap v564: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:27:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:56 smithi098 ceph-mon[34372]: pgmap v565: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:27:56.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:56 smithi027 ceph-mon[30375]: pgmap v565: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:27:56.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:56 smithi089 ceph-mon[34486]: pgmap v565: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:27:58.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:27:58 smithi098 ceph-mon[34372]: pgmap v566: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:27:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:27:58 smithi027 ceph-mon[30375]: pgmap v566: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:27:58.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:27:58 smithi089 ceph-mon[34486]: pgmap v566: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:00.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:00 smithi098 ceph-mon[34372]: pgmap v567: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:00.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:00 smithi027 ceph-mon[30375]: pgmap v567: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:00.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:00 smithi089 ceph-mon[34486]: pgmap v567: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:02.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:02 smithi098 ceph-mon[34372]: pgmap v568: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:28:02.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:02 smithi027 ceph-mon[30375]: pgmap v568: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:28:02.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:02 smithi089 ceph-mon[34486]: pgmap v568: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:28:04.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:04 smithi098 ceph-mon[34372]: pgmap v569: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:28:04.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:04 smithi027 ceph-mon[30375]: pgmap v569: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:28:04.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:04 smithi089 ceph-mon[34486]: pgmap v569: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-21T01:28:06.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:06 smithi098 ceph-mon[34372]: pgmap v570: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:06.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:06 smithi027 ceph-mon[30375]: pgmap v570: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:06.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:06 smithi089 ceph-mon[34486]: pgmap v570: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:08 smithi027 ceph-mon[30375]: pgmap v571: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T01:28:08.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:08 smithi098 ceph-mon[34372]: pgmap v571: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T01:28:08.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:08 smithi089 ceph-mon[34486]: pgmap v571: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-21T01:28:10.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:10 smithi027 ceph-mon[30375]: pgmap v572: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:10.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:10 smithi089 ceph-mon[34486]: pgmap v572: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:10.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:10 smithi098 ceph-mon[34372]: pgmap v572: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:12.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:12 smithi027 ceph-mon[30375]: pgmap v573: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-21T01:28:12.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:12 smithi089 ceph-mon[34486]: pgmap v573: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-21T01:28:12.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:12 smithi098 ceph-mon[34372]: pgmap v573: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-21T01:28:14.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:14 smithi027 ceph-mon[30375]: pgmap v574: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:14.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:14 smithi089 ceph-mon[34486]: pgmap v574: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:14.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:14 smithi098 ceph-mon[34372]: pgmap v574: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-21T01:28:16.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:16 smithi027 ceph-mon[30375]: pgmap v575: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T01:28:16.449 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:16 smithi098 ceph-mon[34372]: pgmap v575: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T01:28:16.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:16 smithi089 ceph-mon[34486]: pgmap v575: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 22 op/s 2024-09-21T01:28:18.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:18 smithi027 ceph-mon[30375]: pgmap v576: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:28:18.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:18 smithi089 ceph-mon[34486]: pgmap v576: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:28:18.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:18 smithi098 ceph-mon[34372]: pgmap v576: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T01:28:20.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:20 smithi027 ceph-mon[30375]: pgmap v577: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:28:20.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:20 smithi089 ceph-mon[34486]: pgmap v577: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:28:20.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:20 smithi098 ceph-mon[34372]: pgmap v577: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:21.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: pgmap v578: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: Unable to set osd_memory_target on smithi098 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: Unable to set osd_memory_target on smithi089 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:28:22.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: pgmap v578: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: Unable to set osd_memory_target on smithi098 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: Unable to set osd_memory_target on smithi089 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:28:22.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:28:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: pgmap v578: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-21T01:28:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi098 to 176.0M 2024-09-21T01:28:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: Adjusting osd_memory_target on smithi089 to 175.9M 2024-09-21T01:28:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: Unable to set osd_memory_target on smithi098 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-21T01:28:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: Unable to set osd_memory_target on smithi089 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-21T01:28:22.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:28:22.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:28:22.682 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:28:24.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:24 smithi089 ceph-mon[34486]: pgmap v579: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:24.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:24 smithi098 ceph-mon[34372]: pgmap v579: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:24.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:24 smithi027 ceph-mon[30375]: pgmap v579: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:26 smithi089 ceph-mon[34486]: pgmap v580: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:26.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:26 smithi098 ceph-mon[34372]: pgmap v580: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:26.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:26 smithi027 ceph-mon[30375]: pgmap v580: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:28 smithi089 ceph-mon[34486]: pgmap v581: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:28.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:28 smithi098 ceph-mon[34372]: pgmap v581: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:28.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:28 smithi027 ceph-mon[30375]: pgmap v581: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:30.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:30 smithi089 ceph-mon[34486]: pgmap v582: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:30.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:30 smithi098 ceph-mon[34372]: pgmap v582: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:30.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:30 smithi027 ceph-mon[30375]: pgmap v582: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:32.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:32 smithi089 ceph-mon[34486]: pgmap v583: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:32.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:32 smithi098 ceph-mon[34372]: pgmap v583: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:32.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:32 smithi027 ceph-mon[30375]: pgmap v583: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:34.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:34 smithi089 ceph-mon[34486]: pgmap v584: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:34.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:34 smithi098 ceph-mon[34372]: pgmap v584: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:34.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:34 smithi027 ceph-mon[30375]: pgmap v584: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:36.484 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:36 smithi027 ceph-mon[30375]: pgmap v585: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:36.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:36 smithi089 ceph-mon[34486]: pgmap v585: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:36.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:36 smithi098 ceph-mon[34372]: pgmap v585: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:38.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:38 smithi089 ceph-mon[34486]: pgmap v586: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:38.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:38 smithi098 ceph-mon[34372]: pgmap v586: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:38.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:38 smithi027 ceph-mon[30375]: pgmap v586: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:40.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:40 smithi089 ceph-mon[34486]: pgmap v587: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:40.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:40 smithi098 ceph-mon[34372]: pgmap v587: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:40.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:40 smithi027 ceph-mon[30375]: pgmap v587: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:42.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:42 smithi089 ceph-mon[34486]: pgmap v588: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:42.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:42 smithi098 ceph-mon[34372]: pgmap v588: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:42.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:42 smithi027 ceph-mon[30375]: pgmap v588: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:44.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:44 smithi089 ceph-mon[34486]: pgmap v589: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:44.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:44 smithi098 ceph-mon[34372]: pgmap v589: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:44.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:44 smithi027 ceph-mon[30375]: pgmap v589: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:46.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:46 smithi089 ceph-mon[34486]: pgmap v590: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:46.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:46 smithi098 ceph-mon[34372]: pgmap v590: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:46.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:46 smithi027 ceph-mon[30375]: pgmap v590: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:48.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:48 smithi089 ceph-mon[34486]: pgmap v591: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:48.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:48 smithi098 ceph-mon[34372]: pgmap v591: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:48.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:48 smithi027 ceph-mon[30375]: pgmap v591: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:50.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:50 smithi089 ceph-mon[34486]: pgmap v592: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:50.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:50 smithi098 ceph-mon[34372]: pgmap v592: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:50.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:50 smithi027 ceph-mon[30375]: pgmap v592: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:52.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:52 smithi089 ceph-mon[34486]: pgmap v593: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:52.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:52 smithi098 ceph-mon[34372]: pgmap v593: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:52.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:52 smithi027 ceph-mon[30375]: pgmap v593: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:28:54.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:54 smithi089 ceph-mon[34486]: pgmap v594: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:54.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:54 smithi098 ceph-mon[34372]: pgmap v594: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:54.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:54 smithi027 ceph-mon[30375]: pgmap v594: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:28:56.499 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:56 smithi089 ceph-mon[34486]: pgmap v595: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:56.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:56 smithi098 ceph-mon[34372]: pgmap v595: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:56.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:56 smithi027 ceph-mon[30375]: pgmap v595: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:28:58.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:28:58 smithi098 ceph-mon[34372]: pgmap v596: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:58.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:28:58 smithi027 ceph-mon[30375]: pgmap v596: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:28:58.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:28:58 smithi089 ceph-mon[34486]: pgmap v596: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:29:00.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:00 smithi098 ceph-mon[34372]: pgmap v597: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:00.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:00 smithi027 ceph-mon[30375]: pgmap v597: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:00.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:00 smithi089 ceph-mon[34486]: pgmap v597: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:02.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:02 smithi098 ceph-mon[34372]: pgmap v598: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:29:02.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:02 smithi027 ceph-mon[30375]: pgmap v598: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:29:02.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:02 smithi089 ceph-mon[34486]: pgmap v598: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:29:04.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:04 smithi098 ceph-mon[34372]: pgmap v599: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:29:04.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:04 smithi027 ceph-mon[30375]: pgmap v599: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:29:04.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:04 smithi089 ceph-mon[34486]: pgmap v599: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:29:06.552 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:06 smithi089 ceph-mon[34486]: pgmap v600: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:29:06.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:06 smithi098 ceph-mon[34372]: pgmap v600: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:29:06.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:06 smithi027 ceph-mon[30375]: pgmap v600: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:29:08.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:08 smithi098 ceph-mon[34372]: pgmap v601: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T01:29:08.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:08 smithi027 ceph-mon[30375]: pgmap v601: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T01:29:08.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:08 smithi089 ceph-mon[34486]: pgmap v601: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T01:29:10.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:10 smithi098 ceph-mon[34372]: pgmap v602: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:29:10.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:10 smithi098 ceph-mon[34372]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T01:29:10.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:10 smithi027 ceph-mon[30375]: pgmap v602: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:29:10.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:10 smithi027 ceph-mon[30375]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T01:29:10.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:10 smithi089 ceph-mon[34486]: pgmap v602: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:29:10.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:10 smithi089 ceph-mon[34486]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T01:29:12.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:12 smithi098 ceph-mon[34372]: pgmap v604: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:29:12.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:12 smithi027 ceph-mon[30375]: pgmap v604: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:29:12.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:12 smithi089 ceph-mon[34486]: pgmap v604: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:29:14.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:14 smithi098 ceph-mon[34372]: pgmap v605: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:29:14.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:14 smithi027 ceph-mon[30375]: pgmap v605: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:29:14.749 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:14 smithi089 ceph-mon[34486]: pgmap v605: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:29:16.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:16 smithi098 ceph-mon[34372]: pgmap v606: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:29:16.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:16 smithi027 ceph-mon[30375]: pgmap v606: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:29:16.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:16 smithi089 ceph-mon[34486]: pgmap v606: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:29:17.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:17 smithi098 ceph-mon[34372]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T01:29:17.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:17 smithi098 ceph-mon[34372]: pgmap v608: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-21T01:29:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:17 smithi027 ceph-mon[30375]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T01:29:17.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:17 smithi027 ceph-mon[30375]: pgmap v608: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-21T01:29:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:17 smithi089 ceph-mon[34486]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T01:29:18.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:17 smithi089 ceph-mon[34486]: pgmap v608: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-21T01:29:20.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:19 smithi089 ceph-mon[34486]: pgmap v609: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 218 B/s wr, 10 op/s 2024-09-21T01:29:20.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:19 smithi098 ceph-mon[34372]: pgmap v609: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 218 B/s wr, 10 op/s 2024-09-21T01:29:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:19 smithi027 ceph-mon[30375]: pgmap v609: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 218 B/s wr, 10 op/s 2024-09-21T01:29:21.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:21 smithi089 ceph-mon[34486]: pgmap v610: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:22.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:21 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:29:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:21 smithi098 ceph-mon[34372]: pgmap v610: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:21 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:29:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:21 smithi027 ceph-mon[30375]: pgmap v610: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:21 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:29:23.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:29:23.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:29:23.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:29:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:29:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:29:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:29:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:29:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:29:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:29:24.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:24 smithi098 ceph-mon[34372]: pgmap v611: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:24.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:24 smithi027 ceph-mon[30375]: pgmap v611: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:24.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:24 smithi089 ceph-mon[34486]: pgmap v611: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:26.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:26 smithi098 ceph-mon[34372]: pgmap v612: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:26.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:26 smithi027 ceph-mon[30375]: pgmap v612: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:26.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:26 smithi089 ceph-mon[34486]: pgmap v612: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:29:28.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:28 smithi098 ceph-mon[34372]: pgmap v613: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:29:28.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:28 smithi027 ceph-mon[30375]: pgmap v613: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:29:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:28 smithi089 ceph-mon[34486]: pgmap v613: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:29:30.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:30 smithi098 ceph-mon[34372]: pgmap v614: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:30.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:30 smithi027 ceph-mon[30375]: pgmap v614: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:30.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:30 smithi089 ceph-mon[34486]: pgmap v614: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:32.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:32 smithi098 ceph-mon[34372]: pgmap v615: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:29:32.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:32 smithi027 ceph-mon[30375]: pgmap v615: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:29:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:32 smithi089 ceph-mon[34486]: pgmap v615: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:29:34.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:34 smithi098 ceph-mon[34372]: pgmap v616: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:34.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:34 smithi027 ceph-mon[30375]: pgmap v616: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:34 smithi089 ceph-mon[34486]: pgmap v616: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:36.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:36 smithi098 ceph-mon[34372]: pgmap v617: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-21T01:29:36.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:36 smithi027 ceph-mon[30375]: pgmap v617: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-21T01:29:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:36 smithi089 ceph-mon[34486]: pgmap v617: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-21T01:29:37.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:37 smithi098 ceph-mon[34372]: pgmap v618: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T01:29:37.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:37 smithi027 ceph-mon[30375]: pgmap v618: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T01:29:38.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:37 smithi089 ceph-mon[34486]: pgmap v618: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T01:29:40.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:39 smithi089 ceph-mon[34486]: pgmap v619: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T01:29:40.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:39 smithi098 ceph-mon[34372]: pgmap v619: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T01:29:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:39 smithi027 ceph-mon[30375]: pgmap v619: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T01:29:42.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:41 smithi089 ceph-mon[34486]: pgmap v620: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:29:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:41 smithi098 ceph-mon[34372]: pgmap v620: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:29:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:41 smithi027 ceph-mon[30375]: pgmap v620: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:29:44.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:43 smithi089 ceph-mon[34486]: pgmap v621: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T01:29:44.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:43 smithi098 ceph-mon[34372]: pgmap v621: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T01:29:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:43 smithi027 ceph-mon[30375]: pgmap v621: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-21T01:29:46.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:45 smithi089 ceph-mon[34486]: pgmap v622: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-21T01:29:46.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:45 smithi098 ceph-mon[34372]: pgmap v622: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-21T01:29:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:45 smithi027 ceph-mon[30375]: pgmap v622: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-21T01:29:47.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:47 smithi089 ceph-mon[34486]: pgmap v623: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:29:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:47 smithi098 ceph-mon[34372]: pgmap v623: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:29:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:47 smithi027 ceph-mon[30375]: pgmap v623: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:29:49.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:49 smithi089 ceph-mon[34486]: pgmap v624: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:49 smithi098 ceph-mon[34372]: pgmap v624: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:50.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:49 smithi027 ceph-mon[30375]: pgmap v624: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:52.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:51 smithi089 ceph-mon[34486]: pgmap v625: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:29:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:51 smithi098 ceph-mon[34372]: pgmap v625: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:29:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:51 smithi027 ceph-mon[30375]: pgmap v625: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:29:54.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:53 smithi089 ceph-mon[34486]: pgmap v626: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:54.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:53 smithi098 ceph-mon[34372]: pgmap v626: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:53 smithi027 ceph-mon[30375]: pgmap v626: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:29:56.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:55 smithi089 ceph-mon[34486]: pgmap v627: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:29:56.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:55 smithi098 ceph-mon[34372]: pgmap v627: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:29:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:55 smithi027 ceph-mon[30375]: pgmap v627: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:29:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:57 smithi089 ceph-mon[34486]: pgmap v628: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:29:58.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:57 smithi098 ceph-mon[34372]: pgmap v628: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:29:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:57 smithi027 ceph-mon[30375]: pgmap v628: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:30:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:29:59 smithi089 ceph-mon[34486]: pgmap v629: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:00.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:29:59 smithi098 ceph-mon[34372]: pgmap v629: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:00.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:29:59 smithi027 ceph-mon[30375]: pgmap v629: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:01.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:00 smithi098 ceph-mon[34372]: overall HEALTH_OK 2024-09-21T01:30:01.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:00 smithi027 ceph-mon[30375]: overall HEALTH_OK 2024-09-21T01:30:01.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:00 smithi089 ceph-mon[34486]: overall HEALTH_OK 2024-09-21T01:30:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:01 smithi098 ceph-mon[34372]: pgmap v630: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:30:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:01 smithi027 ceph-mon[30375]: pgmap v630: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:30:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:01 smithi089 ceph-mon[34486]: pgmap v630: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-21T01:30:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:03 smithi098 ceph-mon[34372]: pgmap v631: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:30:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:03 smithi027 ceph-mon[30375]: pgmap v631: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:30:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:03 smithi089 ceph-mon[34486]: pgmap v631: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:30:06.052 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:05 smithi089 ceph-mon[34486]: pgmap v632: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:30:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:05 smithi098 ceph-mon[34372]: pgmap v632: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:30:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:05 smithi027 ceph-mon[30375]: pgmap v632: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:30:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:07 smithi098 ceph-mon[34372]: pgmap v633: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T01:30:08.182 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:07 smithi027 ceph-mon[30375]: pgmap v633: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T01:30:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:07 smithi089 ceph-mon[34486]: pgmap v633: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T01:30:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:09 smithi098 ceph-mon[34372]: pgmap v634: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:30:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:09 smithi027 ceph-mon[30375]: pgmap v634: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:30:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:09 smithi089 ceph-mon[34486]: pgmap v634: 129 pgs: 129 active+clean; 21 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T01:30:11.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:10 smithi098 ceph-mon[34372]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T01:30:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:10 smithi027 ceph-mon[30375]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T01:30:11.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:10 smithi089 ceph-mon[34486]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T01:30:12.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:11 smithi098 ceph-mon[34372]: pgmap v636: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:30:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:11 smithi027 ceph-mon[30375]: pgmap v636: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:30:12.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:11 smithi089 ceph-mon[34486]: pgmap v636: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:30:14.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:13 smithi098 ceph-mon[34372]: pgmap v637: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:30:14.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:13 smithi027 ceph-mon[30375]: pgmap v637: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:30:14.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:13 smithi089 ceph-mon[34486]: pgmap v637: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:30:16.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:15 smithi098 ceph-mon[34372]: pgmap v638: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:15 smithi027 ceph-mon[30375]: pgmap v638: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:15 smithi089 ceph-mon[34486]: pgmap v638: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:18.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:17 smithi098 ceph-mon[34372]: pgmap v639: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:17 smithi027 ceph-mon[30375]: pgmap v639: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:18.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:17 smithi089 ceph-mon[34486]: pgmap v639: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:19 smithi098 ceph-mon[34372]: pgmap v640: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:19 smithi027 ceph-mon[30375]: pgmap v640: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:19 smithi089 ceph-mon[34486]: pgmap v640: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:30:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:21 smithi098 ceph-mon[34372]: pgmap v641: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 187 B/s wr, 12 op/s 2024-09-21T01:30:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:21 smithi098 ceph-mon[34372]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T01:30:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:21 smithi027 ceph-mon[30375]: pgmap v641: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 187 B/s wr, 12 op/s 2024-09-21T01:30:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:21 smithi027 ceph-mon[30375]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T01:30:22.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:21 smithi089 ceph-mon[34486]: pgmap v641: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 187 B/s wr, 12 op/s 2024-09-21T01:30:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:21 smithi089 ceph-mon[34486]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T01:30:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:22 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:30:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:22 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:30:23.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:22 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:30:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:23 smithi098 ceph-mon[34372]: pgmap v643: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:30:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:30:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:30:24.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:30:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:23 smithi027 ceph-mon[30375]: pgmap v643: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:30:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:30:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:30:24.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:30:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:23 smithi089 ceph-mon[34486]: pgmap v643: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:30:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:30:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:30:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:30:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:25 smithi098 ceph-mon[34372]: pgmap v644: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:30:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:25 smithi027 ceph-mon[30375]: pgmap v644: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:30:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:25 smithi089 ceph-mon[34486]: pgmap v644: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:30:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:27 smithi098 ceph-mon[34372]: pgmap v645: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:27 smithi027 ceph-mon[30375]: pgmap v645: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:27 smithi089 ceph-mon[34486]: pgmap v645: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:30.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:29 smithi098 ceph-mon[34372]: pgmap v646: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:29 smithi027 ceph-mon[30375]: pgmap v646: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:30.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:29 smithi089 ceph-mon[34486]: pgmap v646: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:32.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:31 smithi098 ceph-mon[34372]: pgmap v647: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:31 smithi027 ceph-mon[30375]: pgmap v647: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:32.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:31 smithi089 ceph-mon[34486]: pgmap v647: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:33 smithi098 ceph-mon[34372]: pgmap v648: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T01:30:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:33 smithi027 ceph-mon[30375]: pgmap v648: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T01:30:34.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:33 smithi089 ceph-mon[34486]: pgmap v648: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T01:30:36.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:35 smithi098 ceph-mon[34372]: pgmap v649: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:35 smithi027 ceph-mon[30375]: pgmap v649: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:36.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:35 smithi089 ceph-mon[34486]: pgmap v649: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:38.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:37 smithi098 ceph-mon[34372]: pgmap v650: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-21T01:30:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:37 smithi027 ceph-mon[30375]: pgmap v650: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-21T01:30:38.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:37 smithi089 ceph-mon[34486]: pgmap v650: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-09-21T01:30:40.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:39 smithi098 ceph-mon[34372]: pgmap v651: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T01:30:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:39 smithi027 ceph-mon[30375]: pgmap v651: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T01:30:40.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:39 smithi089 ceph-mon[34486]: pgmap v651: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-21T01:30:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:41 smithi098 ceph-mon[34372]: pgmap v652: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-21T01:30:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:41 smithi027 ceph-mon[30375]: pgmap v652: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-21T01:30:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:41 smithi089 ceph-mon[34486]: pgmap v652: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-21T01:30:44.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:43 smithi098 ceph-mon[34372]: pgmap v653: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:30:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:43 smithi027 ceph-mon[30375]: pgmap v653: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:30:44.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:43 smithi089 ceph-mon[34486]: pgmap v653: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:30:46.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:45 smithi098 ceph-mon[34372]: pgmap v654: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:45 smithi027 ceph-mon[30375]: pgmap v654: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:46.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:45 smithi089 ceph-mon[34486]: pgmap v654: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T01:30:48.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:47 smithi098 ceph-mon[34372]: pgmap v655: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:30:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:47 smithi027 ceph-mon[30375]: pgmap v655: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:30:48.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:47 smithi089 ceph-mon[34486]: pgmap v655: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:30:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:49 smithi098 ceph-mon[34372]: pgmap v656: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:49 smithi027 ceph-mon[30375]: pgmap v656: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:49 smithi089 ceph-mon[34486]: pgmap v656: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:51 smithi098 ceph-mon[34372]: pgmap v657: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:30:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:51 smithi027 ceph-mon[30375]: pgmap v657: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:30:52.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:51 smithi089 ceph-mon[34486]: pgmap v657: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:30:54.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:53 smithi098 ceph-mon[34372]: pgmap v658: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:53 smithi027 ceph-mon[30375]: pgmap v658: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:53 smithi089 ceph-mon[34486]: pgmap v658: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:30:56.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:55 smithi098 ceph-mon[34372]: pgmap v659: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:55 smithi027 ceph-mon[30375]: pgmap v659: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:55 smithi089 ceph-mon[34486]: pgmap v659: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:30:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:57 smithi098 ceph-mon[34372]: pgmap v660: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:30:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:57 smithi027 ceph-mon[30375]: pgmap v660: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:30:58.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:57 smithi089 ceph-mon[34486]: pgmap v660: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:00.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:30:59 smithi098 ceph-mon[34372]: pgmap v661: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:00.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:30:59 smithi027 ceph-mon[30375]: pgmap v661: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:00.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:30:59 smithi089 ceph-mon[34486]: pgmap v661: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:01 smithi098 ceph-mon[34372]: pgmap v662: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:31:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:01 smithi027 ceph-mon[30375]: pgmap v662: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:31:02.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:01 smithi089 ceph-mon[34486]: pgmap v662: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:31:04.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:03 smithi098 ceph-mon[34372]: pgmap v663: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:31:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:03 smithi027 ceph-mon[30375]: pgmap v663: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:31:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:03 smithi089 ceph-mon[34486]: pgmap v663: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:31:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:05 smithi098 ceph-mon[34372]: pgmap v664: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:31:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:05 smithi027 ceph-mon[30375]: pgmap v664: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:31:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:05 smithi089 ceph-mon[34486]: pgmap v664: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:31:08.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:07 smithi098 ceph-mon[34372]: pgmap v665: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T01:31:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:07 smithi027 ceph-mon[30375]: pgmap v665: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T01:31:08.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:07 smithi089 ceph-mon[34486]: pgmap v665: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T01:31:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:09 smithi027 ceph-mon[30375]: pgmap v666: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:31:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:09 smithi089 ceph-mon[34486]: pgmap v666: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:31:10.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:09 smithi098 ceph-mon[34372]: pgmap v666: 129 pgs: 129 active+clean; 20 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:31:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:10 smithi027 ceph-mon[30375]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T01:31:11.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:10 smithi089 ceph-mon[34486]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T01:31:11.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:10 smithi098 ceph-mon[34372]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T01:31:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:11 smithi027 ceph-mon[30375]: pgmap v668: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:31:12.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:11 smithi089 ceph-mon[34486]: pgmap v668: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:31:12.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:11 smithi098 ceph-mon[34372]: pgmap v668: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:31:14.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:13 smithi027 ceph-mon[30375]: pgmap v669: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:31:14.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:13 smithi089 ceph-mon[34486]: pgmap v669: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:31:14.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:13 smithi098 ceph-mon[34372]: pgmap v669: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:31:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:15 smithi027 ceph-mon[30375]: pgmap v670: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:15 smithi089 ceph-mon[34486]: pgmap v670: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:16.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:15 smithi098 ceph-mon[34372]: pgmap v670: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:17 smithi027 ceph-mon[30375]: pgmap v671: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:18.199 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:17 smithi098 ceph-mon[34372]: pgmap v671: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:18.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:17 smithi089 ceph-mon[34486]: pgmap v671: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:19 smithi027 ceph-mon[30375]: pgmap v672: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:20.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:19 smithi089 ceph-mon[34486]: pgmap v672: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:20.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:19 smithi098 ceph-mon[34372]: pgmap v672: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:31:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:21 smithi089 ceph-mon[34486]: pgmap v673: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 189 B/s wr, 12 op/s 2024-09-21T01:31:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:21 smithi089 ceph-mon[34486]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T01:31:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:21 smithi098 ceph-mon[34372]: pgmap v673: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 189 B/s wr, 12 op/s 2024-09-21T01:31:22.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:21 smithi098 ceph-mon[34372]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T01:31:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:21 smithi027 ceph-mon[30375]: pgmap v673: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 189 B/s wr, 12 op/s 2024-09-21T01:31:22.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:21 smithi027 ceph-mon[30375]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T01:31:24.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: pgmap v675: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:31:24.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:31:24.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.193 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.194 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.194 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.194 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.194 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:23 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: pgmap v675: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.229 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:23 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: pgmap v675: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:24.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:23 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: pgmap v676: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:25 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: pgmap v676: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:25 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: pgmap v676: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:25 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:31:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:31:27.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:31:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:31:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:31:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:31:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:31:28.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:28 smithi089 ceph-mon[34486]: pgmap v677: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:28.647 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:28 smithi098 ceph-mon[34372]: pgmap v677: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:28.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:28 smithi027 ceph-mon[30375]: pgmap v677: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:30.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:30 smithi089 ceph-mon[34486]: pgmap v678: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:30.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:30 smithi098 ceph-mon[34372]: pgmap v678: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:30.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:30 smithi027 ceph-mon[30375]: pgmap v678: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:32.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:32 smithi089 ceph-mon[34486]: pgmap v679: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:32.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:32 smithi098 ceph-mon[34372]: pgmap v679: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:32.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:32 smithi027 ceph-mon[30375]: pgmap v679: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:34.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:34 smithi089 ceph-mon[34486]: pgmap v680: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T01:31:34.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:34 smithi098 ceph-mon[34372]: pgmap v680: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T01:31:34.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:34 smithi027 ceph-mon[30375]: pgmap v680: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T01:31:36.500 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:36 smithi089 ceph-mon[34486]: pgmap v681: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:36.648 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:36 smithi098 ceph-mon[34372]: pgmap v681: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:36.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:36 smithi027 ceph-mon[30375]: pgmap v681: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:37.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:37 smithi098 ceph-mon[34372]: pgmap v682: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:37.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:37 smithi027 ceph-mon[30375]: pgmap v682: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:38.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:37 smithi089 ceph-mon[34486]: pgmap v682: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:40.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:39 smithi098 ceph-mon[34372]: pgmap v683: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:39 smithi027 ceph-mon[30375]: pgmap v683: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:40.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:39 smithi089 ceph-mon[34486]: pgmap v683: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:42.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:41 smithi098 ceph-mon[34372]: pgmap v684: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:31:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:41 smithi027 ceph-mon[30375]: pgmap v684: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:31:42.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:41 smithi089 ceph-mon[34486]: pgmap v684: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:31:44.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:43 smithi098 ceph-mon[34372]: pgmap v685: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:43 smithi027 ceph-mon[30375]: pgmap v685: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:44.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:43 smithi089 ceph-mon[34486]: pgmap v685: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:46.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:45 smithi098 ceph-mon[34372]: pgmap v686: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:45 smithi027 ceph-mon[30375]: pgmap v686: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:46.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:45 smithi089 ceph-mon[34486]: pgmap v686: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:48.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:47 smithi098 ceph-mon[34372]: pgmap v687: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:48.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:47 smithi027 ceph-mon[30375]: pgmap v687: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:47 smithi089 ceph-mon[34486]: pgmap v687: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:50.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:49 smithi098 ceph-mon[34372]: pgmap v688: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:49 smithi027 ceph-mon[30375]: pgmap v688: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:49 smithi089 ceph-mon[34486]: pgmap v688: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:52.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:51 smithi098 ceph-mon[34372]: pgmap v689: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:31:52.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:51 smithi027 ceph-mon[30375]: pgmap v689: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:31:52.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:51 smithi089 ceph-mon[34486]: pgmap v689: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-21T01:31:54.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:53 smithi098 ceph-mon[34372]: pgmap v690: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:53 smithi027 ceph-mon[30375]: pgmap v690: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:54.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:53 smithi089 ceph-mon[34486]: pgmap v690: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:31:56.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:55 smithi098 ceph-mon[34372]: pgmap v691: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:56.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:55 smithi027 ceph-mon[30375]: pgmap v691: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:55 smithi089 ceph-mon[34486]: pgmap v691: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T01:31:58.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:57 smithi098 ceph-mon[34372]: pgmap v692: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:58.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:57 smithi027 ceph-mon[30375]: pgmap v692: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:31:58.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:57 smithi089 ceph-mon[34486]: pgmap v692: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-21T01:32:00.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:31:59 smithi098 ceph-mon[34372]: pgmap v693: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:32:00.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:31:59 smithi027 ceph-mon[30375]: pgmap v693: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:32:00.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:31:59 smithi089 ceph-mon[34486]: pgmap v693: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:32:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:01 smithi098 ceph-mon[34372]: pgmap v694: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:32:02.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:01 smithi027 ceph-mon[30375]: pgmap v694: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:32:02.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:01 smithi089 ceph-mon[34486]: pgmap v694: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:32:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:03 smithi098 ceph-mon[34372]: pgmap v695: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:32:04.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:03 smithi027 ceph-mon[30375]: pgmap v695: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:32:04.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:03 smithi089 ceph-mon[34486]: pgmap v695: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-21T01:32:06.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:05 smithi098 ceph-mon[34372]: pgmap v696: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:32:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:05 smithi027 ceph-mon[30375]: pgmap v696: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:32:06.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:05 smithi089 ceph-mon[34486]: pgmap v696: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:32:08.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:07 smithi098 ceph-mon[34372]: pgmap v697: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T01:32:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:07 smithi027 ceph-mon[30375]: pgmap v697: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T01:32:08.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:07 smithi089 ceph-mon[34486]: pgmap v697: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 28 op/s 2024-09-21T01:32:10.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:09 smithi098 ceph-mon[34372]: pgmap v698: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:32:10.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:09 smithi027 ceph-mon[30375]: pgmap v698: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:32:10.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:09 smithi089 ceph-mon[34486]: pgmap v698: 129 pgs: 129 active+clean; 19 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-21T01:32:11.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:10 smithi098 ceph-mon[34372]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T01:32:11.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:10 smithi027 ceph-mon[30375]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T01:32:11.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:10 smithi089 ceph-mon[34486]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T01:32:12.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:11 smithi098 ceph-mon[34372]: pgmap v700: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:32:12.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:11 smithi027 ceph-mon[30375]: pgmap v700: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:32:12.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:11 smithi089 ceph-mon[34486]: pgmap v700: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-21T01:32:14.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:13 smithi098 ceph-mon[34372]: pgmap v701: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:32:14.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:13 smithi027 ceph-mon[30375]: pgmap v701: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:32:14.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:13 smithi089 ceph-mon[34486]: pgmap v701: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 27 op/s 2024-09-21T01:32:16.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:15 smithi098 ceph-mon[34372]: pgmap v702: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:16.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:15 smithi027 ceph-mon[30375]: pgmap v702: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:16.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:15 smithi089 ceph-mon[34486]: pgmap v702: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:18.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:17 smithi098 ceph-mon[34372]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:18.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:17 smithi027 ceph-mon[30375]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:18.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:17 smithi089 ceph-mon[34486]: pgmap v703: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:19.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 -- 172.21.15.27:0/1463239497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9590105ce0 msgr2=0x7f95901060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 --2- 172.21.15.27:0/1463239497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9590105ce0 0x7f95901060f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9580009900 tx=0x7f958002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 -- 172.21.15.27:0/1463239497 shutdown_connections 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 --2- 172.21.15.27:0/1463239497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f959010bf60 0x7f9590112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 --2- 172.21.15.27:0/1463239497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f959010b610 0x7f959010ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 --2- 172.21.15.27:0/1463239497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9590105ce0 0x7f95901060f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 -- 172.21.15.27:0/1463239497 >> 172.21.15.27:0/1463239497 conn(0x7f95901016c0 msgr2=0x7f9590103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 -- 172.21.15.27:0/1463239497 shutdown_connections 2024-09-21T01:32:19.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.578+0000 7f9596200640 1 -- 172.21.15.27:0/1463239497 wait complete. 2024-09-21T01:32:19.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.579+0000 7f9596200640 1 Processor -- start 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.579+0000 7f9596200640 1 -- start start 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.579+0000 7f9596200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 0x7f959019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f9596200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f959010b610 0x7f959019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f9596200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f959010bf60 0x7f95901a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f9596200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9590115c20 con 0x7f959010b610 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f9596200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9590115aa0 con 0x7f959010bf60 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f9596200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9590115da0 con 0x7f9590105ce0 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 0x7f959019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 0x7f959019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41808/0 (socket says 172.21.15.27:41808) 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 -- 172.21.15.27:0/145970295 learned_addr learned my addr 172.21.15.27:0/145970295 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f9587e00640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f959010b610 0x7f959019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:19.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958fe00640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f959010bf60 0x7f95901a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 -- 172.21.15.27:0/145970295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f959010bf60 msgr2=0x7f95901a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f959010bf60 0x7f95901a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 -- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f959010b610 msgr2=0x7f959019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f959010b610 0x7f959019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.580+0000 7f958f400640 1 -- 172.21.15.27:0/145970295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95901a25b0 con 0x7f9590105ce0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.581+0000 7f958f400640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 0x7f959019c680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9580005940 tx=0x7f95800040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.581+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958003f030 con 0x7f9590105ce0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.581+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95800044c0 con 0x7f9590105ce0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.581+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95901a2840 con 0x7f9590105ce0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.581+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580035070 con 0x7f9590105ce0 2024-09-21T01:32:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.581+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95901a9030 con 0x7f9590105ce0 2024-09-21T01:32:19.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.582+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95901060f0 con 0x7f9590105ce0 2024-09-21T01:32:19.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.585+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f9580042030 con 0x7f9590105ce0 2024-09-21T01:32:19.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.586+0000 7f958cc00640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f95740b6680 0x7f95740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:19.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.587+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9578+0+0 (secure 0 0 0) 0x7f9580108160 con 0x7f9590105ce0 2024-09-21T01:32:19.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.587+0000 7f9587e00640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f95740b6680 0x7f95740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:19.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.587+0000 7f9587e00640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f95740b6680 0x7f95740b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95780095f0 tx=0x7f9578005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:19.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.589+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95800d2610 con 0x7f9590105ce0 2024-09-21T01:32:19.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.808+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9590063800 con 0x7f95740b6680 2024-09-21T01:32:19.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.817+0000 7f958cc00640 1 -- 172.21.15.27:0/145970295 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9590063800 con 0x7f95740b6680 2024-09-21T01:32:19.818 INFO:teuthology.orchestra.run.smithi027.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T01:22:17", "first": "2024-09-21T01:23:00", "last": "2024-09-21T01:32:00", "last_pruned": "2024-09-21T01:32:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-21T01:32:19.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.819+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f95740b6680 msgr2=0x7f95740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:19.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.819+0000 7f9596200640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f95740b6680 0x7f95740b8b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f95780095f0 tx=0x7f9578005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:19.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.819+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 msgr2=0x7f959019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.819+0000 7f9596200640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 0x7f959019c680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9580005940 tx=0x7f95800040c0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 shutdown_connections 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f95740b6680 0x7f95740b8b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f959010bf60 0x7f95901a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f959010b610 0x7f959019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 --2- 172.21.15.27:0/145970295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9590105ce0 0x7f959019c680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 >> 172.21.15.27:0/145970295 conn(0x7f95901016c0 msgr2=0x7f9590104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 shutdown_connections 2024-09-21T01:32:19.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:19.820+0000 7f9596200640 1 -- 172.21.15.27:0/145970295 wait complete. 2024-09-21T01:32:19.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T01:32:20.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.069+0000 7f99ce200640 1 -- 172.21.15.27:0/2514218928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c810b610 msgr2=0x7f99c810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.069+0000 7f99ce200640 1 --2- 172.21.15.27:0/2514218928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c810b610 0x7f99c810ba20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009900 tx=0x7f99b402a410 comp rx=0 tx=0).stop 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.069+0000 7f99ce200640 1 -- 172.21.15.27:0/2514218928 shutdown_connections 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.070+0000 7f99ce200640 1 --2- 172.21.15.27:0/2514218928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c810bf60 0x7f99c8112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.070+0000 7f99ce200640 1 --2- 172.21.15.27:0/2514218928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c810b610 0x7f99c810ba20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.070+0000 7f99ce200640 1 --2- 172.21.15.27:0/2514218928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99c8105ce0 0x7f99c81060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.070+0000 7f99ce200640 1 -- 172.21.15.27:0/2514218928 >> 172.21.15.27:0/2514218928 conn(0x7f99c81016c0 msgr2=0x7f99c8103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.070+0000 7f99ce200640 1 -- 172.21.15.27:0/2514218928 shutdown_connections 2024-09-21T01:32:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.070+0000 7f99ce200640 1 -- 172.21.15.27:0/2514218928 wait complete. 2024-09-21T01:32:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 Processor -- start 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 -- start start 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 0x7f99c819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99c810b610 0x7f99c819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c810bf60 0x7f99c81a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8115c70 con 0x7f99c8105ce0 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8115af0 con 0x7f99c810bf60 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99ce200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8115df0 con 0x7f99c810b610 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99c7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 0x7f99c819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99c7400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 0x7f99c819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59892/0 (socket says 172.21.15.27:59892) 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.071+0000 7f99c7400640 1 -- 172.21.15.27:0/1870573788 learned_addr learned my addr 172.21.15.27:0/1870573788 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c6a00640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99c810b610 0x7f99c819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7e00640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c810bf60 0x7f99c81a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7400640 1 -- 172.21.15.27:0/1870573788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99c810b610 msgr2=0x7f99c819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7400640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99c810b610 0x7f99c819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7400640 1 -- 172.21.15.27:0/1870573788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c810bf60 msgr2=0x7f99c81a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7400640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c810bf60 0x7f99c81a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7400640 1 -- 172.21.15.27:0/1870573788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c81a2580 con 0x7f99c8105ce0 2024-09-21T01:32:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99c7400640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 0x7f99c819c650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f99b800e9f0 tx=0x7f99b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:20.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.072+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800cd80 con 0x7f99c8105ce0 2024-09-21T01:32:20.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.073+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c81a2870 con 0x7f99c8105ce0 2024-09-21T01:32:20.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.073+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b80108e0 con 0x7f99c8105ce0 2024-09-21T01:32:20.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.073+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8018980 con 0x7f99c8105ce0 2024-09-21T01:32:20.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.073+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c81a9000 con 0x7f99c8105ce0 2024-09-21T01:32:20.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.074+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c81060f0 con 0x7f99c8105ce0 2024-09-21T01:32:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.076+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f99b8010430 con 0x7f99c8105ce0 2024-09-21T01:32:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.077+0000 7f99bfe00640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99ac0b6700 0x7f99ac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.078+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9578+0+0 (secure 0 0 0) 0x7f99b80e82f0 con 0x7f99c8105ce0 2024-09-21T01:32:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.078+0000 7f99c6a00640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99ac0b6700 0x7f99ac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.079+0000 7f99c6a00640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99ac0b6700 0x7f99ac0b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f99b402aa00 tx=0x7f99b4005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:20.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.080+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99b80b27a0 con 0x7f99c8105ce0 2024-09-21T01:32:20.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:19 smithi098 ceph-mon[34372]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:20.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:19 smithi027 ceph-mon[30375]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:20.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:19 smithi089 ceph-mon[34486]: pgmap v704: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T01:32:20.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.300+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 --> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f99c8063800 con 0x7f99ac0b6700 2024-09-21T01:32:20.309 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.308+0000 7f99bfe00640 1 -- 172.21.15.27:0/1870573788 <== mgr.24110 v2:172.21.15.27:6800/39380413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f99c8063800 con 0x7f99ac0b6700 2024-09-21T01:32:20.309 INFO:teuthology.orchestra.run.smithi027.stdout:/ 1m 6m3h 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99ac0b6700 msgr2=0x7f99ac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99ac0b6700 0x7f99ac0b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f99b402aa00 tx=0x7f99b4005a20 comp rx=0 tx=0).stop 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 msgr2=0x7f99c819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 0x7f99c819c650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f99b800e9f0 tx=0x7f99b800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 shutdown_connections 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f99ac0b6700 0x7f99ac0b8c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99c810bf60 0x7f99c81a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99c810b610 0x7f99c819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 --2- 172.21.15.27:0/1870573788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c8105ce0 0x7f99c819c650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 >> 172.21.15.27:0/1870573788 conn(0x7f99c81016c0 msgr2=0x7f99c8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:20.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 shutdown_connections 2024-09-21T01:32:20.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.311+0000 7f99ce200640 1 -- 172.21.15.27:0/1870573788 wait complete. 2024-09-21T01:32:20.336 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T01:32:20.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T01:32:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.578+0000 7ff036e00640 1 -- 172.21.15.27:0/1970976444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109220 msgr2=0x7ff030109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.578+0000 7ff036e00640 1 --2- 172.21.15.27:0/1970976444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109220 0x7ff030109630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009860 tx=0x7ff01c02a340 comp rx=0 tx=0).stop 2024-09-21T01:32:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 -- 172.21.15.27:0/1970976444 shutdown_connections 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 --2- 172.21.15.27:0/1970976444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109b70 0x7ff0301103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 --2- 172.21.15.27:0/1970976444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109220 0x7ff030109630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 --2- 172.21.15.27:0/1970976444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0301038f0 0x7ff030103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 -- 172.21.15.27:0/1970976444 >> 172.21.15.27:0/1970976444 conn(0x7ff0300ff330 msgr2=0x7ff030101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 -- 172.21.15.27:0/1970976444 shutdown_connections 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.579+0000 7ff036e00640 1 -- 172.21.15.27:0/1970976444 wait complete. 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 Processor -- start 2024-09-21T01:32:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 -- start start 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0301038f0 0x7ff03019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 0x7ff03019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109b70 0x7ff0301a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301137d0 con 0x7ff030109220 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff030113650 con 0x7ff0301038f0 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.580+0000 7ff036e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff030113950 con 0x7ff030109b70 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff035000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 0x7ff03019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109b70 0x7ff0301a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff035000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 0x7ff03019c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59922/0 (socket says 172.21.15.27:59922) 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff035000640 1 -- 172.21.15.27:0/3689814155 learned_addr learned my addr 172.21.15.27:0/3689814155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff035a00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0301038f0 0x7ff03019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 -- 172.21.15.27:0/3689814155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0301038f0 msgr2=0x7ff03019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0301038f0 0x7ff03019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 -- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 msgr2=0x7ff03019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 0x7ff03019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 -- 172.21.15.27:0/3689814155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0301a2360 con 0x7ff030109b70 2024-09-21T01:32:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff035000640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 0x7ff03019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:32:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.581+0000 7ff036400640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109b70 0x7ff0301a1c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff02800ef00 tx=0x7ff02800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.582+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028004b20 con 0x7ff030109b70 2024-09-21T01:32:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.582+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff028003e20 con 0x7ff030109b70 2024-09-21T01:32:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.582+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02800b570 con 0x7ff030109b70 2024-09-21T01:32:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.582+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0301a2650 con 0x7ff030109b70 2024-09-21T01:32:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.582+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0301a8de0 con 0x7ff030109b70 2024-09-21T01:32:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.583+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff030103d00 con 0x7ff030109b70 2024-09-21T01:32:20.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.586+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff02801d020 con 0x7ff030109b70 2024-09-21T01:32:20.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.587+0000 7ff02e000640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff0180b6700 0x7ff0180b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:20.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.587+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9578+0+0 (secure 0 0 0) 0x7ff0280e8430 con 0x7ff030109b70 2024-09-21T01:32:20.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.588+0000 7ff035a00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff0180b6700 0x7ff0180b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:20.588 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.588+0000 7ff035a00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff0180b6700 0x7ff0180b8c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff024009bd0 tx=0x7ff024005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:20.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.589+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff028024030 con 0x7ff030109b70 2024-09-21T01:32:20.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.791+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0301a9080 con 0x7ff030109b70 2024-09-21T01:32:20.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.793+0000 7ff02e000640 1 -- 172.21.15.27:0/3689814155 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+25466 (secure 0 0 0) 0x7ff028004cc0 con 0x7ff030109b70 2024-09-21T01:32:20.794 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:32:20.794 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":97,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:32:09.818762+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff0180b6700 msgr2=0x7ff0180b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff0180b6700 0x7ff0180b8c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff024009bd0 tx=0x7ff024005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109b70 msgr2=0x7ff0301a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109b70 0x7ff0301a1c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff02800ef00 tx=0x7ff02800c740 comp rx=0 tx=0).stop 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 shutdown_connections 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7ff0180b6700 0x7ff0180b8c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff030109b70 0x7ff0301a1c60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff030109220 0x7ff03019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 --2- 172.21.15.27:0/3689814155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0301038f0 0x7ff03019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.795+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 >> 172.21.15.27:0/3689814155 conn(0x7ff0300ff330 msgr2=0x7ff0301078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.796+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 shutdown_connections 2024-09-21T01:32:20.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:20.796+0000 7ff036e00640 1 -- 172.21.15.27:0/3689814155 wait complete. 2024-09-21T01:32:20.820 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T01:13:52.790954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T01:19:02.303884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T01:19:04.941508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T01:19:14.791527+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T01:32:20.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T01:32:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 -- 172.21.15.27:0/3320063200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 msgr2=0x7f6c501095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 --2- 172.21.15.27:0/3320063200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c501095d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009900 tx=0x7f6c3c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 -- 172.21.15.27:0/3320063200 shutdown_connections 2024-09-21T01:32:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 --2- 172.21.15.27:0/3320063200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c50109b10 0x7f6c50110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 --2- 172.21.15.27:0/3320063200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c501095d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 --2- 172.21.15.27:0/3320063200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50103890 0x7f6c50103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.057+0000 7f6c55200640 1 -- 172.21.15.27:0/3320063200 >> 172.21.15.27:0/3320063200 conn(0x7f6c500ff2d0 msgr2=0x7f6c50101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.058+0000 7f6c55200640 1 -- 172.21.15.27:0/3320063200 shutdown_connections 2024-09-21T01:32:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.058+0000 7f6c55200640 1 -- 172.21.15.27:0/3320063200 wait complete. 2024-09-21T01:32:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 Processor -- start 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 -- start start 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c50103890 0x7f6c5019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c5019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50109b10 0x7f6c501a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50113780 con 0x7f6c501091c0 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50113600 con 0x7f6c50109b10 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.059+0000 7f6c55200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50113900 con 0x7f6c50103890 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c5019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c5019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59938/0 (socket says 172.21.15.27:59938) 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 -- 172.21.15.27:0/4126286218 learned_addr learned my addr 172.21.15.27:0/4126286218 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4f400640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c50103890 0x7f6c5019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4fe00640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50109b10 0x7f6c501a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 -- 172.21.15.27:0/4126286218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c50103890 msgr2=0x7f6c5019c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c50103890 0x7f6c5019c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 -- 172.21.15.27:0/4126286218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50109b10 msgr2=0x7f6c501a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50109b10 0x7f6c501a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.060+0000 7f6c4ea00640 1 -- 172.21.15.27:0/4126286218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c501a22f0 con 0x7f6c501091c0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.061+0000 7f6c4ea00640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c5019c960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02a910 tx=0x7f6c3c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.061+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c03f030 con 0x7f6c501091c0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.061+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c004490 con 0x7f6c501091c0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.061+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c035070 con 0x7f6c501091c0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.061+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c501a2580 con 0x7f6c501091c0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.061+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501a8d70 con 0x7f6c501091c0 2024-09-21T01:32:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.062+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c50103ca0 con 0x7f6c501091c0 2024-09-21T01:32:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.065+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6c3c042030 con 0x7f6c501091c0 2024-09-21T01:32:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.067+0000 7f6c47e00640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c340b6700 0x7f6c340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.067+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9578+0+0 (secure 0 0 0) 0x7f6c3c1081a0 con 0x7f6c501091c0 2024-09-21T01:32:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.067+0000 7f6c4f400640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c340b6700 0x7f6c340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.068+0000 7f6c4f400640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c340b6700 0x7f6c340b8c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c400095f0 tx=0x7f6c40005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.068+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6c3c0d2650 con 0x7f6c501091c0 2024-09-21T01:32:21.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:20 smithi098 ceph-mon[34372]: from='client.25852 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:32:21.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:20 smithi098 ceph-mon[34372]: from='client.15819 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:32:21.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:20 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3689814155' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:32:21.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:20 smithi027 ceph-mon[30375]: from='client.25852 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:32:21.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:20 smithi027 ceph-mon[30375]: from='client.15819 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:32:21.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:20 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3689814155' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:32:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:20 smithi089 ceph-mon[34486]: from='client.25852 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:32:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:20 smithi089 ceph-mon[34486]: from='client.15819 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T01:32:21.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:20 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3689814155' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:32:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.266+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6c501a9060 con 0x7f6c501091c0 2024-09-21T01:32:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.266+0000 7f6c47e00640 1 -- 172.21.15.27:0/4126286218 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f6c3c0d7590 con 0x7f6c501091c0 2024-09-21T01:32:21.267 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c340b6700 msgr2=0x7f6c340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c340b6700 0x7f6c340b8c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c400095f0 tx=0x7f6c40005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 msgr2=0x7f6c5019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c5019c960 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02a910 tx=0x7f6c3c031a20 comp rx=0 tx=0).stop 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 shutdown_connections 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c340b6700 0x7f6c340b8c10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c50109b10 0x7f6c501a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c501091c0 0x7f6c5019c960 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 --2- 172.21.15.27:0/4126286218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c50103890 0x7f6c5019c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 >> 172.21.15.27:0/4126286218 conn(0x7f6c500ff2d0 msgr2=0x7f6c50108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 shutdown_connections 2024-09-21T01:32:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.269+0000 7f6c55200640 1 -- 172.21.15.27:0/4126286218 wait complete. 2024-09-21T01:32:21.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 -- 172.21.15.27:0/1654370230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c105af0 msgr2=0x7fae7c105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 --2- 172.21.15.27:0/1654370230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c105af0 0x7fae7c105f00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009900 tx=0x7fae6c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 -- 172.21.15.27:0/1654370230 shutdown_connections 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 --2- 172.21.15.27:0/1654370230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae7c10bd70 0x7fae7c1125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 --2- 172.21.15.27:0/1654370230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 0x7fae7c10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 --2- 172.21.15.27:0/1654370230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c105af0 0x7fae7c105f00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 -- 172.21.15.27:0/1654370230 >> 172.21.15.27:0/1654370230 conn(0x7fae7c101530 msgr2=0x7fae7c103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 -- 172.21.15.27:0/1654370230 shutdown_connections 2024-09-21T01:32:21.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.528+0000 7fae7b200640 1 -- 172.21.15.27:0/1654370230 wait complete. 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.529+0000 7fae7b200640 1 Processor -- start 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.529+0000 7fae7b200640 1 -- start start 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae7c105af0 0x7fae7c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 0x7fae7c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 0x7fae7c1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c115a30 con 0x7fae7c10bd70 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c1158b0 con 0x7fae7c10b420 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c115bb0 con 0x7fae7c105af0 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 0x7fae7c1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 0x7fae7c1a1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59952/0 (socket says 172.21.15.27:59952) 2024-09-21T01:32:21.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae7a800640 1 -- 172.21.15.27:0/815579875 learned_addr learned my addr 172.21.15.27:0/815579875 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae79e00640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae7c105af0 0x7fae7c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.530+0000 7fae79400640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 0x7fae7c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae79400640 1 -- 172.21.15.27:0/815579875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae7c105af0 msgr2=0x7fae7c19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae79400640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae7c105af0 0x7fae7c19c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae79400640 1 -- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 msgr2=0x7fae7c1a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae79400640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 0x7fae7c1a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae79400640 1 -- 172.21.15.27:0/815579875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c1a2300 con 0x7fae7c10b420 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae7a800640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 0x7fae7c1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae79400640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 0x7fae7c19c970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06c790 tx=0x7fae6800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6800cd00 con 0x7fae7c10b420 2024-09-21T01:32:21.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae680035f0 con 0x7fae7c10b420 2024-09-21T01:32:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c1a25f0 con 0x7fae7c10b420 2024-09-21T01:32:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68010690 con 0x7fae7c10b420 2024-09-21T01:32:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.531+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c1a8d80 con 0x7fae7c10b420 2024-09-21T01:32:21.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.532+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c105f00 con 0x7fae7c10b420 2024-09-21T01:32:21.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.535+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fae680037a0 con 0x7fae7c10b420 2024-09-21T01:32:21.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.537+0000 7fae72a00640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fae600b6700 0x7fae600b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:21.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.537+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9578+0+0 (secure 0 0 0) 0x7fae680e7f60 con 0x7fae7c10b420 2024-09-21T01:32:21.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.537+0000 7fae79e00640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fae600b6700 0x7fae600b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:21.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.538+0000 7fae79e00640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fae600b6700 0x7fae600b8c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae6c031440 tx=0x7fae6c009560 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:21.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.538+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fae68056710 con 0x7fae7c10b420 2024-09-21T01:32:21.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.743+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae7c02e640 con 0x7fae7c10b420 2024-09-21T01:32:21.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.744+0000 7fae72a00640 1 -- 172.21.15.27:0/815579875 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fae680ae030 con 0x7fae7c10b420 2024-09-21T01:32:21.745 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T01:32:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fae600b6700 msgr2=0x7fae600b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fae600b6700 0x7fae600b8c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fae6c031440 tx=0x7fae6c009560 comp rx=0 tx=0).stop 2024-09-21T01:32:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 msgr2=0x7fae7c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:21.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 0x7fae7c19c970 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06c790 tx=0x7fae6800ee50 comp rx=0 tx=0).stop 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 shutdown_connections 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fae600b6700 0x7fae600b8c10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae7c10bd70 0x7fae7c1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae7c10b420 0x7fae7c19c970 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 --2- 172.21.15.27:0/815579875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae7c105af0 0x7fae7c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 >> 172.21.15.27:0/815579875 conn(0x7fae7c101530 msgr2=0x7fae7c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 shutdown_connections 2024-09-21T01:32:21.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:21.746+0000 7fae7b200640 1 -- 172.21.15.27:0/815579875 wait complete. 2024-09-21T01:32:21.768 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 -- 172.21.15.27:0/2194834643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f741093c0 msgr2=0x7f2f741097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 --2- 172.21.15.27:0/2194834643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f741093c0 0x7f2f741097d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2f64009900 tx=0x7f2f6402a410 comp rx=0 tx=0).stop 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 -- 172.21.15.27:0/2194834643 shutdown_connections 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 --2- 172.21.15.27:0/2194834643 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f74109d10 0x7f2f74110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 --2- 172.21.15.27:0/2194834643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f741093c0 0x7f2f741097d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 --2- 172.21.15.27:0/2194834643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f74103a90 0x7f2f74103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 -- 172.21.15.27:0/2194834643 >> 172.21.15.27:0/2194834643 conn(0x7f2f740ff4d0 msgr2=0x7f2f74101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 -- 172.21.15.27:0/2194834643 shutdown_connections 2024-09-21T01:32:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.007+0000 7f2f7c800640 1 -- 172.21.15.27:0/2194834643 wait complete. 2024-09-21T01:32:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.008+0000 7f2f7c800640 1 Processor -- start 2024-09-21T01:32:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.008+0000 7f2f7c800640 1 -- start start 2024-09-21T01:32:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7c800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f74103a90 0x7f2f7419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7c800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 0x7f2f7419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7c800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f74109d10 0x7f2f741a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7c800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f74113a20 con 0x7f2f74109d10 2024-09-21T01:32:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7c800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2f741138a0 con 0x7f2f741093c0 2024-09-21T01:32:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7c800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f74113ba0 con 0x7f2f74103a90 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f79400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 0x7f2f7419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f7a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f74109d10 0x7f2f741a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f79e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f74103a90 0x7f2f7419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f79400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 0x7f2f7419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:49070/0 (socket says 172.21.15.27:49070) 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.009+0000 7f2f79400640 1 -- 172.21.15.27:0/547084128 learned_addr learned my addr 172.21.15.27:0/547084128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f79400640 1 -- 172.21.15.27:0/547084128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f74103a90 msgr2=0x7f2f7419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f79400640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f74103a90 0x7f2f7419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f79400640 1 -- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f74109d10 msgr2=0x7f2f741a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f79400640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f74109d10 0x7f2f741a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f79400640 1 -- 172.21.15.27:0/547084128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741a25f0 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f7a800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f74109d10 0x7f2f741a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f79400640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 0x7f2f7419cc60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005ab0 tx=0x7f2f64003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6403f070 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741a2880 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f64031b50 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64035030 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.010+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f741a9070 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.011+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f74104d00 con 0x7f2f741093c0 2024-09-21T01:32:22.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.014+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2f64042070 con 0x7f2f741093c0 2024-09-21T01:32:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.016+0000 7f2f6ea00640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2f5c0b6680 0x7f2f5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.016+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7f2f641080a0 con 0x7f2f741093c0 2024-09-21T01:32:22.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.016+0000 7f2f79e00640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2f5c0b6680 0x7f2f5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.017+0000 7f2f79e00640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2f5c0b6680 0x7f2f5c0b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009bd0 tx=0x7f2f68005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:22.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.017+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f640d25a0 con 0x7f2f741093c0 2024-09-21T01:32:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:21 smithi098 ceph-mon[34372]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 188 B/s wr, 12 op/s 2024-09-21T01:32:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/4126286218' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:32:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:21 smithi098 ceph-mon[34372]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T01:32:22.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:21 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/815579875' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T01:32:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:21 smithi027 ceph-mon[30375]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 188 B/s wr, 12 op/s 2024-09-21T01:32:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/4126286218' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:32:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:21 smithi027 ceph-mon[30375]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T01:32:22.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:21 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/815579875' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T01:32:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.214+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f7419d7e0 con 0x7f2f741093c0 2024-09-21T01:32:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.215+0000 7f2f6ea00640 1 -- 172.21.15.27:0/547084128 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f2f640d74e0 con 0x7f2f741093c0 2024-09-21T01:32:22.215 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-09-21T01:32:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2f5c0b6680 msgr2=0x7f2f5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2f5c0b6680 0x7f2f5c0b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009bd0 tx=0x7f2f68005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 msgr2=0x7f2f7419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 0x7f2f7419cc60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005ab0 tx=0x7f2f64003b90 comp rx=0 tx=0).stop 2024-09-21T01:32:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 shutdown_connections 2024-09-21T01:32:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f2f5c0b6680 0x7f2f5c0b8b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f74109d10 0x7f2f741a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2f741093c0 0x7f2f7419cc60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 --2- 172.21.15.27:0/547084128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f74103a90 0x7f2f7419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.217+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 >> 172.21.15.27:0/547084128 conn(0x7f2f740ff4d0 msgr2=0x7f2f741069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.218+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 shutdown_connections 2024-09-21T01:32:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.218+0000 7f2f7c800640 1 -- 172.21.15.27:0/547084128 wait complete. 2024-09-21T01:32:22.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T01:32:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:21 smithi089 ceph-mon[34486]: pgmap v705: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 188 B/s wr, 12 op/s 2024-09-21T01:32:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/4126286218' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T01:32:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:21 smithi089 ceph-mon[34486]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T01:32:22.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:21 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/815579875' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T01:32:22.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 -- 172.21.15.27:0/870477377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4114370 msgr2=0x7f51a4116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 --2- 172.21.15.27:0/870477377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4114370 0x7f51a4116840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f519c009ca0 tx=0x7f519c02a8c0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 -- 172.21.15.27:0/870477377 shutdown_connections 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 --2- 172.21.15.27:0/870477377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4114370 0x7f51a4116840 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 --2- 172.21.15.27:0/870477377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 --2- 172.21.15.27:0/870477377 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4077390 0x7f51a40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 -- 172.21.15.27:0/870477377 >> 172.21.15.27:0/870477377 conn(0x7f51a40ff4f0 msgr2=0x7f51a4101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 -- 172.21.15.27:0/870477377 shutdown_connections 2024-09-21T01:32:22.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.472+0000 7f51a9c00640 1 -- 172.21.15.27:0/870477377 wait complete. 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.473+0000 7f51a9c00640 1 Processor -- start 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.473+0000 7f51a9c00640 1 -- start start 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.473+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a41a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.473+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 0x7f51a41a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.473+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4114370 0x7f51a41a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a9c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4119fd0 con 0x7f51a4077390 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a9c00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4119e50 con 0x7f51a4114370 2024-09-21T01:32:22.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a9c00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f51a411a150 con 0x7f51a4075dd0 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f519be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 0x7f51a41a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a41a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f519be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 0x7f51a41a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60002/0 (socket says 172.21.15.27:60002) 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a41a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41912/0 (socket says 172.21.15.27:41912) 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f519be00640 1 -- 172.21.15.27:0/2862805932 learned_addr learned my addr 172.21.15.27:0/2862805932 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a3400640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4114370 0x7f51a41a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 -- 172.21.15.27:0/2862805932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4114370 msgr2=0x7f51a41a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4114370 0x7f51a41a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 -- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 msgr2=0x7f51a41a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 0x7f51a41a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a2a00640 1 -- 172.21.15.27:0/2862805932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41a6860 con 0x7f51a4075dd0 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f51a3400640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4114370 0x7f51a41a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.474+0000 7f519be00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 0x7f51a41a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:32:22.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.475+0000 7f51a2a00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a41a0930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9f0 tx=0x7f519400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.475+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519400ccc0 con 0x7f51a4075dd0 2024-09-21T01:32:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.475+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41a6b50 con 0x7f51a4075dd0 2024-09-21T01:32:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.475+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51940108e0 con 0x7f51a4075dd0 2024-09-21T01:32:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.475+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41ad2e0 con 0x7f51a4075dd0 2024-09-21T01:32:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.475+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194018750 con 0x7f51a4075dd0 2024-09-21T01:32:22.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.476+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a4076cc0 con 0x7f51a4075dd0 2024-09-21T01:32:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.479+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5194010430 con 0x7f51a4075dd0 2024-09-21T01:32:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.480+0000 7f519b400640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f51880b6680 0x7f51880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.480+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7f51940e80a0 con 0x7f51a4075dd0 2024-09-21T01:32:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.481+0000 7f519be00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f51880b6680 0x7f51880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.481+0000 7f519be00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f51880b6680 0x7f51880b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f518c0095f0 tx=0x7f518c005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:22.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.482+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51940ae220 con 0x7f51a4075dd0 2024-09-21T01:32:22.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.681+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f51a402e640 con 0x7f51a4075dd0 2024-09-21T01:32:22.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.683+0000 7f519b400640 1 -- 172.21.15.27:0/2862805932 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f51940b2600 con 0x7f51a4075dd0 2024-09-21T01:32:22.683 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f51880b6680 msgr2=0x7f51880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f51880b6680 0x7f51880b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f518c0095f0 tx=0x7f518c005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 msgr2=0x7f51a41a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a41a0930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f519400e9f0 tx=0x7f519400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 shutdown_connections 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f51880b6680 0x7f51880b8b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f51a4114370 0x7f51a41a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a4077390 0x7f51a41a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 --2- 172.21.15.27:0/2862805932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a4075dd0 0x7f51a41a0930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 >> 172.21.15.27:0/2862805932 conn(0x7f51a40ff4f0 msgr2=0x7f51a4110270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 shutdown_connections 2024-09-21T01:32:22.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.685+0000 7f51a9c00640 1 -- 172.21.15.27:0/2862805932 wait complete. 2024-09-21T01:32:22.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T01:32:22.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 -- 172.21.15.27:0/1178318255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410b400 msgr2=0x7f135410b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 --2- 172.21.15.27:0/1178318255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410b400 0x7f135410b810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1344009900 tx=0x7f134402a410 comp rx=0 tx=0).stop 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 -- 172.21.15.27:0/1178318255 shutdown_connections 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 --2- 172.21.15.27:0/1178318255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f135410bd50 0x7f1354112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 --2- 172.21.15.27:0/1178318255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410b400 0x7f135410b810 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 --2- 172.21.15.27:0/1178318255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f1354105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 -- 172.21.15.27:0/1178318255 >> 172.21.15.27:0/1178318255 conn(0x7f13541014f0 msgr2=0x7f1354103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 -- 172.21.15.27:0/1178318255 shutdown_connections 2024-09-21T01:32:22.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.938+0000 7f135b200640 1 -- 172.21.15.27:0/1178318255 wait complete. 2024-09-21T01:32:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 Processor -- start 2024-09-21T01:32:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 -- start start 2024-09-21T01:32:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f135419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f135410b400 0x7f135419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410bd50 0x7f13541a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1354115a10 con 0x7f135410bd50 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1354115890 con 0x7f135410b400 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.939+0000 7f135b200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1354115b90 con 0x7f1354105ad0 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f135419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f135a800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410bd50 0x7f13541a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f135419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:41920/0 (socket says 172.21.15.27:41920) 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 -- 172.21.15.27:0/2971010319 learned_addr learned my addr 172.21.15.27:0/2971010319 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359400640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f135410b400 0x7f135419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 -- 172.21.15.27:0/2971010319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f135410b400 msgr2=0x7f135419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f135410b400 0x7f135419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 -- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410bd50 msgr2=0x7f13541a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410bd50 0x7f13541a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.940+0000 7f1359e00640 1 -- 172.21.15.27:0/2971010319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13541a2380 con 0x7f1354105ad0 2024-09-21T01:32:22.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.941+0000 7f1359e00640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f135419c450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f134800e9f0 tx=0x7f134800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:22.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.941+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134800cd80 con 0x7f1354105ad0 2024-09-21T01:32:22.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.941+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480054d0 con 0x7f1354105ad0 2024-09-21T01:32:22.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.941+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541a2670 con 0x7f1354105ad0 2024-09-21T01:32:22.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.941+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348010690 con 0x7f1354105ad0 2024-09-21T01:32:22.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.941+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13541a8e00 con 0x7f1354105ad0 2024-09-21T01:32:22.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.942+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1354105ee0 con 0x7f1354105ad0 2024-09-21T01:32:22.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.944+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f1348004e90 con 0x7f1354105ad0 2024-09-21T01:32:22.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.946+0000 7f134ea00640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f133c0b6680 0x7f133c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:22.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.946+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7f13480e7f20 con 0x7f1354105ad0 2024-09-21T01:32:22.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.946+0000 7f1359400640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f133c0b6680 0x7f133c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:22.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.947+0000 7f1359400640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f133c0b6680 0x7f133c0b8b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1344009230 tx=0x7f1344031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:22.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:22.950+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f13480ae040 con 0x7f1354105ad0 2024-09-21T01:32:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:22 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/547084128' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T01:32:23.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:22 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2862805932' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T01:32:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:22 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/547084128' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T01:32:23.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:22 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2862805932' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T01:32:23.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.200+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f135402e640 con 0x7f1354105ad0 2024-09-21T01:32:23.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.209+0000 7f134ea00640 1 -- 172.21.15.27:0/2971010319 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+834047 (secure 0 0 0) 0x7f13480b2420 con 0x7f1354105ad0 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 26, 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "active_gid": 24110, 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "active_name": "x", 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addrs": { 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 39380413 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 39380413 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "active_addr": "172.21.15.27:6801/39380413", 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "active_change": "2024-09-21T01:11:40.307023+0000", 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T01:32:23.222 INFO:teuthology.orchestra.run.smithi027.stdout: "available": true, 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "standbys": [ 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24190, 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "z", 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540719139924082687, 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.227 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T01:32:23.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T01:32:23.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T01:32:23.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T01:32:23.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T01:32:23.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.249 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T01:32:23.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T01:32:23.257 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.259 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.266 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T01:32:23.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T01:32:23.268 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T01:32:23.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T01:32:23.270 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T01:32:23.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T01:32:23.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T01:32:23.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T01:32:23.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T01:32:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.287 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T01:32:23.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T01:32:23.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.299 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.303 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T01:32:23.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T01:32:23.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T01:32:23.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T01:32:23.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T01:32:23.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T01:32:23.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.318 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.319 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.321 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.325 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.326 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.331 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.336 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.345 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.354 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.355 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T01:32:23.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T01:32:23.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T01:32:23.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T01:32:23.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T01:32:23.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T01:32:23.361 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T01:32:23.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T01:32:23.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T01:32:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T01:32:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T01:32:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.382 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T01:32:23.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.387 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.393 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.395 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T01:32:23.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T01:32:23.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T01:32:23.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T01:32:23.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.400 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.401 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.407 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.411 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.416 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.420 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T01:32:23.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.424 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.426 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.427 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.428 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.433 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T01:32:23.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T01:32:23.438 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.442 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.443 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.445 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.448 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.449 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.454 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T01:32:23.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T01:32:23.456 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.460 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.463 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.464 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.465 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.467 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.468 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.471 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.476 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.477 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.483 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.484 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.485 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.486 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.491 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T01:32:23.494 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.496 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.499 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T01:32:23.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.506 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.509 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.510 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.514 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.516 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.517 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.523 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T01:32:23.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T01:32:23.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.536 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.542 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.545 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T01:32:23.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.550 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T01:32:23.551 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.553 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.555 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.556 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.557 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.563 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.565 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T01:32:23.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:23.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.572 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.574 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.584 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.590 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.591 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.593 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.606 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.610 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.614 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.617 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.621 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.623 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.632 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.635 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.640 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.641 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.649 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.650 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T01:32:23.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T01:32:23.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T01:32:23.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T01:32:23.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.659 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.660 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.661 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.662 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.663 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.664 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.665 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.666 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.667 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.668 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.671 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.672 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.673 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.674 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.675 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.676 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T01:32:23.677 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.678 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.679 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.680 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.681 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.682 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.683 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.684 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.685 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.686 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.687 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.688 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.689 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.691 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.692 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.693 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:23.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "gid": 24200, 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "y", 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr_features": 4540719139924082687, 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.695 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.696 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.697 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.699 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.700 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.701 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.702 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T01:32:23.703 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.704 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.706 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.708 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.710 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.711 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T01:32:23.712 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.713 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T01:32:23.714 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.718 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.720 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.721 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.723 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T01:32:23.724 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.725 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.726 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.727 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T01:32:23.728 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.729 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T01:32:23.730 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T01:32:23.731 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T01:32:23.732 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.734 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T01:32:23.735 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.736 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.737 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.738 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.739 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.740 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T01:32:23.741 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.742 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.743 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.744 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.745 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.746 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T01:32:23.747 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.748 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.749 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.750 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.752 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T01:32:23.753 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.754 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T01:32:23.755 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T01:32:23.756 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.757 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.758 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T01:32:23.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.760 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.761 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.762 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.763 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.764 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T01:32:23.765 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.766 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.767 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.768 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.769 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.770 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.771 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.772 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.773 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.774 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.775 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.776 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.778 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.779 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T01:32:23.780 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.781 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.782 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.783 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.785 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.786 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T01:32:23.787 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.788 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.789 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T01:32:23.790 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.791 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.792 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.799 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.800 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.802 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.803 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.804 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T01:32:23.805 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.806 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.807 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.808 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.810 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.811 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.812 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.813 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.814 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.815 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.816 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.817 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.818 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.819 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.821 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.823 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.824 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T01:32:23.825 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T01:32:23.826 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T01:32:23.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T01:32:23.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T01:32:23.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T01:32:23.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.831 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T01:32:23.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T01:32:23.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.834 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.835 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.837 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.838 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T01:32:23.839 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.841 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.842 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.843 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.844 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.845 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.846 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.847 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.848 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.849 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.850 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.851 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.852 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.853 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.854 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.855 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.856 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.857 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.858 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.859 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.860 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.861 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.863 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.864 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T01:32:23.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T01:32:23.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.867 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.868 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.869 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.870 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.871 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.872 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.873 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.874 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.876 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.877 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.878 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.879 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.880 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T01:32:23.881 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:23.882 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.883 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.884 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T01:32:23.885 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.886 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.887 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.888 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.889 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.890 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T01:32:23.891 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.892 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.893 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.894 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.896 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.897 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.898 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.899 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.900 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.901 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.902 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.903 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.904 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.905 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.906 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.907 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.908 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.910 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.911 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.913 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.914 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.915 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.916 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.917 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.918 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.919 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.920 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.921 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.922 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.923 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.924 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.927 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.928 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.929 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.930 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.931 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.931 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.934 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T01:32:23.936 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T01:32:23.937 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.938 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.939 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.942 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.943 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.945 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.946 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.947 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T01:32:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.951 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.952 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.953 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.954 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.955 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.956 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.960 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.961 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.962 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.963 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.964 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.965 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.966 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.967 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.968 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.970 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.971 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.972 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.973 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.974 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.975 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.976 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.980 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.981 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.982 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.983 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.984 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.985 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.986 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.991 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.992 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:23.993 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:23.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.995 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.996 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.997 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:23.998 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:23.999 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.000 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.003 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.005 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.006 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.007 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.008 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.011 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.012 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.013 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.014 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.015 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.016 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.017 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.018 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.019 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.020 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T01:32:24.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T01:32:24.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.023 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.026 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T01:32:24.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T01:32:24.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.031 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.034 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.035 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.036 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.037 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.038 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.039 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.041 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.042 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.043 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.044 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.045 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.046 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.047 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T01:32:24.048 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:24.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:24.050 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.051 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.053 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.054 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.055 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.056 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.057 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.058 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.061 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.062 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.063 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.064 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.065 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.066 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T01:32:24.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.068 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.070 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.071 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.072 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.074 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.079 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.080 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.081 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.082 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.083 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.084 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.085 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.086 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.087 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.088 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.089 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.090 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.091 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.092 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.094 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.095 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.096 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.097 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.098 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.099 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.101 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.104 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.105 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.106 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.108 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.110 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.112 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.113 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.115 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T01:32:24.116 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.117 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.118 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.119 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.120 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.121 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.122 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.123 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.124 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.125 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.126 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.127 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.128 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.129 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.130 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.131 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.132 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.133 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.134 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.135 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.136 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.137 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.138 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.140 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.141 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.142 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.143 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.145 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.147 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.148 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.149 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.150 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.151 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.152 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.153 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.154 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.155 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.156 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.157 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.158 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.159 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.160 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.161 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T01:32:24.162 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.163 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T01:32:24.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T01:32:24.165 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.168 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.169 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.170 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.171 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.172 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T01:32:24.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T01:32:24.174 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T01:32:24.175 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.176 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "modules": [ 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard", 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "iostat", 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "nfs", 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "restful", 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_schedule" 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "available_modules": [ 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "alerts", 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.177 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.178 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.179 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.180 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.181 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_destination": { 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_destination", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.184 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_from_name": { 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_from_name", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "Ceph", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Email From: name", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_host": { 2024-09-21T01:32:24.185 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_host", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP server", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_password": { 2024-09-21T01:32:24.186 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_password", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Password to authenticate with", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_port": { 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_port", 2024-09-21T01:32:24.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "465", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP port", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_sender": { 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_sender", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.188 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SMTP envelope sender", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_ssl": { 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_ssl", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.189 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "smtp_user": { 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smtp_user", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.190 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "User to authenticate as", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.191 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "balancer", 2024-09-21T01:32:24.192 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "active": { 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "active", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.193 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_time": { 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_time", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0000", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T01:32:24.194 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "begin_weekday": { 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "begin_weekday", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T01:32:24.195 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_max_iterations": { 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_max_iterations", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "25", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "250", 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.196 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_metrics": { 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_metrics", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.197 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_compat_step": { 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crush_compat_step", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.5", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0.001", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "0.999", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "aggressiveness of optimization", 2024-09-21T01:32:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "end_time": { 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_time", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2359", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T01:32:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "end_weekday": { 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "end_weekday", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "0", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "6", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T01:32:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.204 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min_score": { 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_score", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.206 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "upmap", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-compat", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "none", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "read", 2024-09-21T01:32:24.207 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap", 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap-read" 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Balancer mode", 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_ids": { 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_ids", 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.208 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.209 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_deviation": { 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_deviation", 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.211 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.214 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.214 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.214 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "upmap_max_optimizations": { 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "upmap_max_optimizations", 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:24.215 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_down_multiplier": { 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_down_multiplier", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_refresh_rate": { 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_refresh_rate", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "20", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "agent_starting_port": { 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "agent_starting_port", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4721", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_ptrace": { 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_ptrace", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_interval": { 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_interval", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "autotune_memory_target_ratio": { 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0.7", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm_log_destination": { 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephadm_log_destination", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "file", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "file,syslog", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "syslog" 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T01:32:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "cgroups_split": { 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cgroups_split", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.224 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "config_checks_enabled": { 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_checks_enabled", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.225 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "config_dashboard": { 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_dashboard", 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.226 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_alertmanager": { 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_alertmanager", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.227 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_base": { 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_base", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.228 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Container image name, without the tag", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_elasticsearch": { 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_elasticsearch", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.229 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "elasticsearch container image", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_grafana": { 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_grafana", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.230 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_haproxy": { 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_haproxy", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.231 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "HAproxy container image", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_agent": { 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_agent", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.232 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger agent container image", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_collector": { 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_collector", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.233 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger collector container image", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_jaeger_query": { 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_jaeger_query", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.234 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Jaeger query container image", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_keepalived": { 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_keepalived", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.235 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Keepalived container image", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_loki": { 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_loki", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Loki container image", 2024-09-21T01:32:24.236 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nginx": { 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nginx", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nginx container image", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.237 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_node_exporter": { 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_node_exporter", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.238 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_nvmeof": { 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_nvmeof", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Nvme-of container image", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.239 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_oauth2_proxy": { 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "oauth2-proxy container image", 2024-09-21T01:32:24.240 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_prometheus": { 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_prometheus", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Prometheus container image", 2024-09-21T01:32:24.241 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_promtail": { 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_promtail", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Promtail container image", 2024-09-21T01:32:24.242 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba": { 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB container image", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.243 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_samba_metrics": { 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_samba_metrics", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.244 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "container_image_snmp_gateway": { 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_image_snmp_gateway", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "SNMP Gateway container image", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "container_init": { 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "container_init", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.246 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "daemon_cache_timeout": { 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "daemon_cache_timeout", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_cephadm_command_timeout": { 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "900", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.248 INFO:teuthology.orchestra.run.smithi027.stdout: "default_registry": { 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "default_registry", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "docker.io", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "device_cache_timeout": { 2024-09-21T01:32:24.249 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_cache_timeout", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1800", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache device inventory", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "device_enhanced_scan": { 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_enhanced_scan", 2024-09-21T01:32:24.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "facts_cache_timeout": { 2024-09-21T01:32:24.251 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "facts_cache_timeout", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "seconds to cache host facts data", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "grafana_dashboards_path": { 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "grafana_dashboards_path", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.252 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "host_check_interval": { 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "host_check_interval", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.253 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to perform a host check", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "hw_monitoring": { 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hw_monitoring", 2024-09-21T01:32:24.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "inventory_list_all": { 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "inventory_list_all", 2024-09-21T01:32:24.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.257 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "log_refresh_metadata": { 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_refresh_metadata", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.258 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.259 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.260 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.261 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T01:32:24.262 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T01:32:24.263 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "*", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "max_count_per_host": { 2024-09-21T01:32:24.264 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_count_per_host", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of daemons per service per host", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd_draining_count": { 2024-09-21T01:32:24.265 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_osd_draining_count", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "migration_current": { 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "migration_current", 2024-09-21T01:32:24.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "internal - do not modify", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "mode": { 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mode", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.267 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "root", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm-package", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "root" 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.268 INFO:teuthology.orchestra.run.smithi027.stdout: "oob_default_addr": { 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "oob_default_addr", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "169.254.1.1", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "prometheus_alerts_path": { 2024-09-21T01:32:24.269 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus_alerts_path", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.270 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_insecure": { 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_insecure", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_password": { 2024-09-21T01:32:24.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_password", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_url": { 2024-09-21T01:32:24.272 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_url", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "registry_username": { 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "registry_username", 2024-09-21T01:32:24.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "secure_monitoring_stack": { 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secure_monitoring_stack", 2024-09-21T01:32:24.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "service_discovery_port": { 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "service_discovery_port", 2024-09-21T01:32:24.275 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8765", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "cephadm service discovery port", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.276 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_config_file": { 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_config_file", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.277 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_count_max": { 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.278 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "ssh_keepalive_interval": { 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssh_keepalive_interval", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.279 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "use_agent": { 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_agent", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.280 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "use_repo_digest": { 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "use_repo_digest", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.281 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_failed_host_check": { 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_failed_host_check", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.282 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_daemons": { 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_daemons", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.283 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_on_stray_hosts": { 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_on_stray_hosts", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.284 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crash", 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.285 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.286 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.287 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.288 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.289 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "retain_interval": { 2024-09-21T01:32:24.290 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retain_interval", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "31536000", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.291 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_recent_interval": { 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_recent_interval", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.292 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1209600", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dashboard", 2024-09-21T01:32:24.293 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.294 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.295 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.296 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_ENABLED": { 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.303 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T01:32:24.304 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T01:32:24.305 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T01:32:24.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T01:32:24.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.308 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_URL": { 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_URL", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.309 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.310 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.311 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.312 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.313 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.314 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "[]", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.315 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "{}", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.316 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.317 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.318 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.319 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.320 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.321 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.322 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.323 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.324 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.325 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.326 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T01:32:24.327 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.328 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8", 2024-09-21T01:32:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "45", 2024-09-21T01:32:24.330 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.331 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.332 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "admin", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.333 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.334 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.335 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.336 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.337 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.338 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "cross_origin_url": { 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cross_origin_url", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.339 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "crt_file": { 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "crt_file", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.340 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "debug": { 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "debug", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Enable/disable debug options", 2024-09-21T01:32:24.341 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "jwt_token_ttl": { 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "jwt_token_ttl", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "28800", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.342 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.343 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.344 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.345 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.346 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.350 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.350 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.350 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.350 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.351 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "motd": { 2024-09-21T01:32:24.352 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "motd", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "The message of the day", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect_resolve_ip_addr": { 2024-09-21T01:32:24.353 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:24.354 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:24.355 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8080", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.356 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T01:32:24.357 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl_server_port": { 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl_server_port", 2024-09-21T01:32:24.358 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8443", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "sso_oauth2": { 2024-09-21T01:32:24.359 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sso_oauth2", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T01:32:24.360 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "redirect", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "redirect" 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.361 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.362 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "url_prefix": { 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url_prefix", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.363 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_monitoring": { 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_monitoring", 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.364 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "monitor device health metrics", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.365 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.368 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.369 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "mark_out_threshold": { 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mark_out_threshold", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "2419200", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.370 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": { 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pool_name", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "device_health_metrics", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.371 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "retention_period": { 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "retention_period", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15552000", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.372 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long to retain device health metrics", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_frequency": { 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_frequency", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.373 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "self_heal": { 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "self_heal", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.374 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.375 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.376 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "warn_threshold": { 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "warn_threshold", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7257600", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.377 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "diskprediction_local", 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.379 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.381 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.382 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "predict_interval": { 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predict_interval", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "86400", 2024-09-21T01:32:24.383 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "predictor_model": { 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "predictor_model", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "prophetstor", 2024-09-21T01:32:24.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "600", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.385 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "influx", 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": false, 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "influxdb python module not found", 2024-09-21T01:32:24.387 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "batch_size": { 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "batch_size", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5000", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "database": { 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "database", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "ceph", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.389 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": { 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "hostname", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server hostname", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.390 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "30", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "5", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.391 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.392 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.393 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.394 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.395 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "password": { 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "password", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.396 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "password of InfluxDB server user", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "port": { 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "port", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.397 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "8086", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "InfluxDB server port", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.398 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "ssl": { 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ssl", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.399 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "false", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "threads": { 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threads", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.400 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "32", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "username": { 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "username", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.401 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "username of InfluxDB server user", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "verify_ssl": { 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "verify_ssl", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "true", 2024-09-21T01:32:24.402 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "insights", 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.403 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.404 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.405 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.406 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.407 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.408 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "iostat", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.410 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.411 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.412 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.413 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.414 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.415 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "k8sevents", 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_event_retention_days": { 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "ceph_event_retention_days", 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.416 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "7", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "config_check_secs": { 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "config_check_secs", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.417 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "10", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.418 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.419 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.421 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.422 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.423 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "localpool", 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.424 INFO:teuthology.orchestra.run.smithi027.stdout: "failure_domain": { 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "failure_domain", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "host", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "failure domain for any created local pool", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.425 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.426 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.427 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.428 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.429 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": { 2024-09-21T01:32:24.430 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "min_size", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default min_size for any created local pool", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "num_rep": { 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "num_rep", 2024-09-21T01:32:24.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default replica count for any created local pool", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": { 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_num", 2024-09-21T01:32:24.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "128", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "prefix": { 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prefix", 2024-09-21T01:32:24.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "name prefix for any created local pool", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "subtree": { 2024-09-21T01:32:24.435 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "subtree", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "rack", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mds_autoscaler", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.437 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.438 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.439 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.440 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.441 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.442 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirroring", 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.445 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.446 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.447 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.448 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "nfs", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.449 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.450 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.451 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.452 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.453 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.454 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "fail_fs": { 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "fail_fs", 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.456 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.457 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.458 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.460 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.461 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator": { 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "orchestrator", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "cephadm", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "rook", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "test_orchestrator" 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Orchestrator backend", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.463 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_perf_query", 2024-09-21T01:32:24.464 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.465 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.466 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.467 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.468 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.469 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.470 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "osd_support", 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.471 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.472 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.473 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.474 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.475 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.476 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "pg_autoscaler", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.477 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.478 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.479 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.480 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.481 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.482 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.483 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.484 INFO:teuthology.orchestra.run.smithi027.stdout: "threshold": { 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "threshold", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "3.0", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1.0", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "scaling threshold", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "progress", 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_pg_recovery_event": { 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_pg_recovery_event", 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.486 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.487 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.491 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.492 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max_completed_events": { 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_completed_events", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "50", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.493 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "number of past completed events to remember", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "sleep_interval": { 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sleep_interval", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.494 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "how long the module is going to sleep", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.495 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "prometheus", 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "cache": { 2024-09-21T01:32:24.496 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cache", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "exclude_perf_counters": { 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "exclude_perf_counters", 2024-09-21T01:32:24.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.498 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.499 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.500 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.501 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.502 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.503 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools": { 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.504 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "300", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.505 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "scrape_interval": { 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "scrape_interval", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15.0", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.506 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "::", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T01:32:24.507 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "9283", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T01:32:24.508 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.509 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "stale_cache_strategy": { 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stale_cache_strategy", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "log", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.510 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_behaviour": { 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_behaviour", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "default", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.511 INFO:teuthology.orchestra.run.smithi027.stdout: "default", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "error" 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "standby_error_status_code": { 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "standby_error_status_code", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "400", 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "599", 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.513 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.514 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.516 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_snap_create": { 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_snap_create", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "mirror_snapshot_schedule": { 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.520 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "trash_purge_schedule": { 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "trash_purge_schedule", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "restful", 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "enable_auth": { 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enable_auth", 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.523 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "key_file": { 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "key_file", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.524 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.525 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.526 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.527 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.528 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.529 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "max_requests": { 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_requests", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.530 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "500", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "server_addr": { 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_addr", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.531 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "server_port": { 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "server_port", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.532 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rgw", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.535 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.538 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.539 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "5", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T01:32:24.540 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.541 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rook", 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.542 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.543 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.546 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.547 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "storage_class": { 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "storage_class", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "local", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.548 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "selftest", 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.549 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.550 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.551 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.552 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.553 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "roption1": { 2024-09-21T01:32:24.554 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption1", 2024-09-21T01:32:24.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.555 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.555 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.555 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.555 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.555 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.557 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.557 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.557 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.557 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "roption2": { 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "roption2", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "xyz", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.558 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption1": { 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption1", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.559 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption2": { 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption2", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.560 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption3": { 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption3", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "float", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.561 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption4": { 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption4", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.562 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption5": { 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption5", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.563 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption6": { 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption6", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.564 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "rwoption7": { 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rwoption7", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "1", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "42", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.565 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.566 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "testkey": { 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testkey", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.567 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "testlkey": { 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testlkey", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "testnewline": { 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "testnewline", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "smb", 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "internal_store_backend": { 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "internal_store_backend", 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.570 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.571 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.574 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.575 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "update_orchestration": { 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "update_orchestration", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.577 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.578 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_m_granularity": { 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "allow_m_granularity", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "dump_on_update": { 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "dump_on_update", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "dump database to debug log on update", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.581 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.582 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.583 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "stats", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.587 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.590 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.592 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "status", 2024-09-21T01:32:24.593 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.594 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.596 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.597 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.598 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.599 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telegraf", 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "address": { 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "address", 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.600 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.601 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "15", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.602 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.603 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.604 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.605 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.606 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.608 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "telemetry", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_basic": { 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_basic", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T01:32:24.609 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_crash": { 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_crash", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T01:32:24.610 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_device": { 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_device", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.611 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_ident": { 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_ident", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T01:32:24.612 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "channel_perf": { 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "channel_perf", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.613 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "contact": { 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "contact", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.614 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "description": { 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "description", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.615 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "device_url": { 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "device_url", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.616 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enabled": { 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "enabled", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "24", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "8", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.618 INFO:teuthology.orchestra.run.smithi027.stdout: "last_opt_revision": { 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "last_opt_revision", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "1", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard": { 2024-09-21T01:32:24.619 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "leaderboard_description": { 2024-09-21T01:32:24.620 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "leaderboard_description", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.622 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.623 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.624 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.626 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "organization": { 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "organization", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.627 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "proxy": { 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "proxy", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.628 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.629 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "url": { 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "url", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.630 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "test_orchestrator", 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.631 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.632 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.633 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.634 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.635 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.636 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.637 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.638 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.639 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.640 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.641 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.642 INFO:teuthology.orchestra.run.smithi027.stdout: "max_concurrent_clones": { 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "max_concurrent_clones", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "4", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "periodic_async_work": { 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "periodic_async_work", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Periodically check for async work", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_delay": { 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_delay", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.645 INFO:teuthology.orchestra.run.smithi027.stdout: "snapshot_clone_no_wait": { 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "True", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "can_run": true, 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "error_string": "", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "module_options": { 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "discovery_interval": { 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "discovery_interval", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "uint", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "100", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.648 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "identifier": { 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "identifier", 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.649 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "interval": { 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "interval", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "secs", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "60", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.650 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "log_level": { 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_level", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.651 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster": { 2024-09-21T01:32:24.652 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_cluster_level": { 2024-09-21T01:32:24.653 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_cluster_level", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "info", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [ 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "critical", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "debug", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "error", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "info", 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: "warning" 2024-09-21T01:32:24.654 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "log_to_file": { 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "log_to_file", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "bool", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "False", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.655 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "sqlite3_killpoint": { 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "sqlite3_killpoint", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "dev", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "0", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.656 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_host": { 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_host", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.657 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_port": { 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_port", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "int", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "10051", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.658 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "zabbix_sender": { 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "zabbix_sender", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "str", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "level": "advanced", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 0, 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "min": "", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "max": "", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "enum_allowed": [], 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "desc": "", 2024-09-21T01:32:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: "long_desc": "", 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "tags": [], 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "see_also": [] 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "services": { 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "dashboard": "https://172.21.15.27:8443/" 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "always_on_modules": { 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "octopus": [ 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T01:32:24.660 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "pacific": [ 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T01:32:24.661 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "quincy": [ 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T01:32:24.662 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "reef": [ 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2024-09-21T01:32:24.663 INFO:teuthology.orchestra.run.smithi027.stdout: "squid": [ 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "balancer", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "crash", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "devicehealth", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "orchestrator", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscaler", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "progress", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "rbd_support", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "status", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "telemetry", 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "volumes" 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "last_failure_osd_epoch": 5, 2024-09-21T01:32:24.664 INFO:teuthology.orchestra.run.smithi027.stdout: "active_clients": [ 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "devicehealth", 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1753540833 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "libcephsqlite", 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1131893290 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "rbd_support", 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3904109230 2024-09-21T01:32:24.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 479718636 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.667 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "snap_schedule", 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2383155355 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "volumes", 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2024-09-21T01:32:24.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2024-09-21T01:32:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3404006079 2024-09-21T01:32:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:24.669 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-09-21T01:32:24.669 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:32:24.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.217+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f133c0b6680 msgr2=0x7f133c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:24.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.217+0000 7f135b200640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f133c0b6680 0x7f133c0b8b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1344009230 tx=0x7f1344031400 comp rx=0 tx=0).stop 2024-09-21T01:32:24.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.217+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 msgr2=0x7f135419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:24.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.217+0000 7f135b200640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f135419c450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f134800e9f0 tx=0x7f134800edf0 comp rx=0 tx=0).stop 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 shutdown_connections 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f133c0b6680 0x7f133c0b8b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f135410bd50 0x7f13541a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f135410b400 0x7f135419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 --2- 172.21.15.27:0/2971010319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1354105ad0 0x7f135419c450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 >> 172.21.15.27:0/2971010319 conn(0x7f13541014f0 msgr2=0x7f1354103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 shutdown_connections 2024-09-21T01:32:24.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:23.218+0000 7f135b200640 1 -- 172.21.15.27:0/2971010319 wait complete. 2024-09-21T01:32:24.677 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T01:32:24.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T01:32:24.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:23 smithi027 ceph-mon[30375]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:32:24.678 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:23 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2971010319' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T01:32:24.679 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:23 smithi098 ceph-mon[34372]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:32:24.679 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:23 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2971010319' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T01:32:24.679 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:22 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/547084128' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T01:32:24.679 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:22 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2862805932' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T01:32:24.680 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:23 smithi089 ceph-mon[34486]: pgmap v707: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-21T01:32:24.680 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:23 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2971010319' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T01:32:25.078 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:25.843 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 224, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 218, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 993264, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 134304, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 0, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.058015318, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.026976864, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.038416093, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:25.844 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 217, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.023861589, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000109961 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 993264, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 134304 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:25.845 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 1, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 166, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 160, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 109504, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 3, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.033340372, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.028523249, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002192279, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 164, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.020966875, 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000127846 2024-09-21T01:32:25.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 109504 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 4, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 4, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3040, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 448, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 2, 2024-09-21T01:32:25.847 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.003249357, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000041736, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.003236097, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.006117901, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002039300 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.848 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T01:32:25.849 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:25.850 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:25.851 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T01:32:25.852 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 149998, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 30, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:25.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 968, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 2, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:25.854 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 225, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 961597, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 225, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 225, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 961597, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:25.855 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:25.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:25.856 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:32:26.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:25 smithi098 ceph-mon[34372]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T01:32:26.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:25 smithi027 ceph-mon[30375]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T01:32:26.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:25 smithi089 ceph-mon[34486]: pgmap v708: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-21T01:32:26.805 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-21T01:32:26.805 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T01:32:26.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T01:32:27.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:26 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:32:27.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:26 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:32:27.246 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:27.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:26 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 749, 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 751, 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 198244, 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 238246, 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 6, 2024-09-21T01:32:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.099862271, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.043723225, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.022733861, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.021976736, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 749, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.039299234, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000052468 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 653, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 821, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 162630, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 570453, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.127754628, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.068830555, 2024-09-21T01:32:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.023478106, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.026279846, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 820, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.067111245, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000081842 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 364384 2024-09-21T01:32:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 989, 2024-09-21T01:32:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 981, 2024-09-21T01:32:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1282608, 2024-09-21T01:32:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 376692, 2024-09-21T01:32:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 3, 2024-09-21T01:32:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.192018914, 2024-09-21T01:32:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.098765251, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.068510722, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.026643216, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 980, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.096451882, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000098420 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1033296, 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 140736 2024-09-21T01:32:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3573, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.113558523, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000031782 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-09-21T01:32:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 1, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-09-21T01:32:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-09-21T01:32:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-09-21T01:32:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_write_failed": 0 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-09-21T01:32:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 5, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 2, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 4195735, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 4195424, 2024-09-21T01:32:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 5 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 3, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 3, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 0, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 1, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 1, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 0, 2024-09-21T01:32:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 115620, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 316420096 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-09-21T01:32:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.053 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.056 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.057 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:28.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.059 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 2, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 1, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 1882, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.061 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:28.062 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:28.063 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 191198, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 53, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 196, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 8672, 2024-09-21T01:32:28.065 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 13, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2139, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2139, 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-09-21T01:32:28.066 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2139, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.346965636, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000629717 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2139, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2140 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2139, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2139, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1388, 2024-09-21T01:32:28.067 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 749, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:28.068 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-09-21T01:32:28.069 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 98, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 10, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 5, 2024-09-21T01:32:28.070 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 5, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:28.071 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 2387, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1412934, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 2387, 2024-09-21T01:32:28.072 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2387, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1412934, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:28.073 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2139, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 1436, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2139, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1436, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.074 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2139, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2139, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2139, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2139, 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:28.075 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:28.076 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x55fbce0976c0": { 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "trackedop": { 2024-09-21T01:32:28.078 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-09-21T01:32:28.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:28.079 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:32:28.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:27 smithi098 ceph-mon[34372]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:27 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:32:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:27 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:32:28.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:27 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:32:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:27 smithi027 ceph-mon[30375]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:27 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:32:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:27 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:32:28.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:27 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:32:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:27 smithi089 ceph-mon[34486]: pgmap v709: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:27 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T01:32:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:27 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T01:32:28.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:27 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' 2024-09-21T01:32:28.971 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-09-21T01:32:28.972 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-09-21T01:32:28.972 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-09-21T01:32:28.972 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-09-21T01:32:28.972 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-09-21T01:32:28.972 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:28.972 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:28.972 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T01:32:28.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T01:32:29.364 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:30.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:29 smithi098 ceph-mon[34372]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:30.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:29 smithi027 ceph-mon[30375]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 522, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 577, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 163935, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 215046, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 8, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.100571109, 2024-09-21T01:32:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.045742316, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.047876330, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.005839552, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 576, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.086353216, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000149918 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1031, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000288718, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000280 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:30.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 27, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 208, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 7204, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 401123, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 11, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.044146760, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.029928534, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.010780482, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000368733, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 206, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.025402025, 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000123310 2024-09-21T01:32:30.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 33, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000021391, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000648 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 367424 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 294, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 278, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1063375, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 202614, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 9, 2024-09-21T01:32:30.190 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.059909698, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.023434494, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.049257546, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000272984, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 277, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.041599261, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000150177 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000526, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000131 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.191 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1060576, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 143616 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 49, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.002036293, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000041557 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.192 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000024682, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000012341 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 2, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.193 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 2, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 11, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 16, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 16, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 13, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 4, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:30.194 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 3, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 21, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 15, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 2, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 4, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:30.195 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 4, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 234, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:30.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 14, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 9, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 10, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-09-21T01:32:30.198 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 5, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 36, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_write_failed": 0 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-09-21T01:32:30.199 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 5, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 5, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 2, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 2, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 4195735, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-09-21T01:32:30.200 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.008450089, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002112522 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 20, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 22, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 2, 2024-09-21T01:32:30.201 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 19, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 23, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 4, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 16, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 20, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 4, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 116604, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 325865472 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 2, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:30.202 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 2, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 12, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.204 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-09-21T01:32:30.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.206 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.209 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.210 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.211 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 2, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 1, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 612, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:30.215 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:30.216 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:30.217 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 246069, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 76, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 196, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:30.218 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 87541, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 396, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 40, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 11733, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 33, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 40, 2024-09-21T01:32:30.219 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 40, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.243993721, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.006099843 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 40, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 62 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 40, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 0, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 40, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-09-21T01:32:30.220 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 4, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 7, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 6, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:30.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-09-21T01:32:30.222 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 98, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 10, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 9, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 9, 2024-09-21T01:32:30.223 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 14, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 1 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-09-21T01:32:30.224 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 839, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1141686, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 839, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 839, 2024-09-21T01:32:30.225 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1141686, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 40, 2024-09-21T01:32:30.226 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 11733, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 40, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 11733, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:30.227 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 40, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 40, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 40, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 40, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T01:32:30.228 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x561ce6833450": { 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 5, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1431, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 5, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 4, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1431, 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:30.230 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: "trackedop": { 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 33 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:30.231 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:32:30.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:29 smithi089 ceph-mon[34486]: pgmap v710: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=14 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:31.368 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:31.368 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T01:32:31.369 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T01:32:31.755 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:32.147 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:31 smithi098 ceph-mon[34372]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:32.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:31 smithi027 ceph-mon[30375]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:32.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:31 smithi089 ceph-mon[34486]: pgmap v711: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:32.551 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 994, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 986, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 1284160, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 378926, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.187014214, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.081337737, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.075866073, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.030442034, 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:32.552 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 985, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.104202592, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000105789 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1033840, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 142016 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 574, 2024-09-21T01:32:32.553 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 746, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 141770, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 549850, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.114043468, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.059120585, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.020504633, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.025364118, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 744, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.058769196, 2024-09-21T01:32:32.554 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000078990 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 367568 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 754, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 753, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 200652, 2024-09-21T01:32:32.555 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 238631, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.108746925, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.044546797, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.026609455, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.028232923, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 752, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.053054988, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000070551 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.556 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3445, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.129165877, 2024-09-21T01:32:32.557 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000037493 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-09-21T01:32:32.558 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-09-21T01:32:32.559 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 1, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:32.560 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:32.561 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:32.562 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:32.563 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:32.564 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_started": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "uninline_write_failed": 0 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 3, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-09-21T01:32:32.565 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 2, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 4195283, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 4194892, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.566 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 3 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 3, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 3, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 0, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 1, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 1, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 0, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 0, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 0, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 0, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-09-21T01:32:32.567 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 115832, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 316424192 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.570 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.571 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.572 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.573 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.574 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.575 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.576 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.577 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:32.579 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 1, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 0, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 1, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 1887, 2024-09-21T01:32:32.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:32.581 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:32.582 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 190414, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 51, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:32.583 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 196, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 8672, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 13, 2024-09-21T01:32:32.584 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2067, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2067, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2067, 2024-09-21T01:32:32.585 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.156944786, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001043514 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2067, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2068 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2067, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2067, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1315, 2024-09-21T01:32:32.586 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 750, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:32.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-09-21T01:32:32.588 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 98, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 10, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 5, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 5, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-09-21T01:32:32.589 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-09-21T01:32:32.590 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 2318, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 1402140, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 2318, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:32.591 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2318, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 1402140, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:32.592 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2067, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 984, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2067, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 984, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:32.593 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2067, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2067, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2067, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2067, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:32.595 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x558d1a191450": { 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-09-21T01:32:32.596 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "trackedop": { 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-09-21T01:32:32.597 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-09-21T01:32:33.589 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-09-21T01:32:33.589 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-09-21T01:32:33.589 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=0 2024-09-21T01:32:33.590 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=0 2024-09-21T01:32:33.590 INFO:tasks.check_counter:Daemon mds.j mds.root_rsnaps=0 2024-09-21T01:32:33.590 INFO:tasks.check_counter:Daemon mds.j mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:33.590 INFO:tasks.check_counter:Daemon mds.j mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:33.590 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T01:32:33.590 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T01:32:33.848 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:32:34.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:33 smithi098 ceph-mon[34372]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:34.154 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:33 smithi089 ceph-mon[34486]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:34.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:33 smithi027 ceph-mon[30375]: pgmap v712: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 3516, 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2848, 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1152801, 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 2672935, 2024-09-21T01:32:34.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 13, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.356563379, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.132042703, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.151832398, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.081410574, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2853, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.258225000, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000090509 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2864, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000639856, 2024-09-21T01:32:34.417 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000223 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 305856, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 87648 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 2742, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2665, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1709431, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 1018603, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 14, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 6, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.258827615, 2024-09-21T01:32:34.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.114278217, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.111106186, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.046942152, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2667, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.408830138, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000153292 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1036, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000238864, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000230 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1073552, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 146864 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 3295, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2795, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1028768, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 1531268, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 34, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 10, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.371500578, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.137281567, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.138465777, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.096535202, 2024-09-21T01:32:34.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2784, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.358714877, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000128848 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1678, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000528247, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000314 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 100320, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 422432 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-09-21T01:32:34.421 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 615, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.126565899, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000205798 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 4263, 2024-09-21T01:32:34.422 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.130858618, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000030696 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 181, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 181, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 181, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 25.166940559, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.139043870 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-09-21T01:32:34.423 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 12, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 36, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 36, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 35, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 2, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 2, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 2, 2024-09-21T01:32:34.424 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 249, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 231, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:34.425 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 4, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 234, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 6, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:34.426 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 37, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 142, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 191, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 144, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 7, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 30 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 20, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2024-09-21T01:32:34.427 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 40, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 2, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 5, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 3, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 5, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 65, 2024-09-21T01:32:34.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_started": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_write_failed": 0 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 296, 2024-09-21T01:32:34.429 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 296, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 2, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-09-21T01:32:34.430 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 5857788, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 0, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 262, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.702062653, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002679628 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 0 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 29, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 32, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 3, 2024-09-21T01:32:34.431 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 20, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 20, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 0, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 36, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 42, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 6, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 2, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 22, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 20, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 123112, 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 338444288 2024-09-21T01:32:34.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 237, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 1, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 181, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 34, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 2, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 7, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.007380845, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001054406 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 110, 2024-09-21T01:32:34.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.024164831, 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000219680 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.434 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 8, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.003288080, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000411010 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.435 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.436 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 10, 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.001715178, 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000171517 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 8, 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.015692819, 2024-09-21T01:32:34.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001961602 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 10, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.053971804, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.005397180 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.439 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000977048, 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000488524 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.001120733, 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000560366 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-09-21T01:32:34.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.002280499, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.001140249 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.012643687, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.006321843 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 4, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.020099798, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.005024949 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.442 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.001406016, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000703008 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:34.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 11, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.028680340, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002607303 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.444 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 24.993518881, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 8.331172960 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.445 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 2, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 5, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 3, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 2, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 27, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 13, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 596, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T01:32:34.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:34.447 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:34.448 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 319424, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 116, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:34.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 196, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 154710, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 1512, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-09-21T01:32:34.450 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 5233, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 1683146, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 37, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 5233, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 5233, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 70.932375970, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.013554820 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 5233, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 5295 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 5233, 2024-09-21T01:32:34.451 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 3, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 5230, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 11, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 3, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 265, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 37, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 4868, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-09-21T01:32:34.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 22, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-09-21T01:32:34.453 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 98, 2024-09-21T01:32:34.454 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 19, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 12, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 12, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 74, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 0, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 15 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 1, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-09-21T01:32:34.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 3585, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 1, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 4356, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 3, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 9549, 2024-09-21T01:32:34.456 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 2984304, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 9549, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 9549, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2984304, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:34.457 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 5233, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1683505, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 5233, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1683505, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 5233, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 5233, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 5233, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 5233, 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:34.459 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 3, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 359, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 3, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:34.460 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 3, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 359, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55aa42a1b450": { 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:34.461 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 296, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1663484, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 296, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 262, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1663484, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:34.462 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout: "trackedop": { 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 59 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:34.463 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T01:32:35.401 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T01:32:35.401 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T01:32:35.401 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=20 2024-09-21T01:32:35.402 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-21T01:32:35.402 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T01:32:35.402 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-21T01:32:35.402 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-21T01:32:35.402 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T01:32:35.402 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T01:32:35.648 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:32:36.075 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:35 smithi089 ceph-mon[34486]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T01:32:36.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:35 smithi027 ceph-mon[30375]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T01:32:36.198 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T01:32:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1626, 2024-09-21T01:32:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1627, 2024-09-21T01:32:36.198 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 2143651, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 516760, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 6, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.183795210, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.091071975, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.036557820, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.042418221, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1625, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.061300002, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000037723 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:36.199 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 257, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 247, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1038424, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 143322, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:36.200 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.047611883, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.020743851, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.037467166, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000102668, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 247, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.020661189, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000083648 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.201 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1036768, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 141984 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 751, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 922, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 184731, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 605909, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:36.202 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.093228826, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.043158787, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.016417145, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.026152346, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 920, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.032308587, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000035118 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.203 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 367408 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3990, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.103837033, 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000026024 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.204 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 0, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.205 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 29, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 29, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 29, 2024-09-21T01:32:36.206 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 2, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2024-09-21T01:32:36.207 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2024-09-21T01:32:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2024-09-21T01:32:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:36.208 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 4, 2024-09-21T01:32:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 234, 2024-09-21T01:32:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 6, 2024-09-21T01:32:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:36.209 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:36.210 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2024-09-21T01:32:36.211 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:36.212 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_started": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_write_failed": 0 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 296, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2024-09-21T01:32:36.213 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 5857788, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 5857652, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 296 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.214 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 20, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 23, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 3, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 11, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 11, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 0, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 29, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 29, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 0, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 120152, 2024-09-21T01:32:36.215 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 316420096 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.216 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.217 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-09-21T01:32:36.218 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-09-21T01:32:36.219 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-09-21T01:32:36.220 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-09-21T01:32:36.221 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.222 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.223 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.224 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.225 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.226 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:36.227 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 2, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 5, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 3, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 3, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2024-09-21T01:32:36.228 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 3043, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:36.229 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:36.230 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 189938, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 65, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:36.231 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 196, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 112399, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 1034, 2024-09-21T01:32:36.232 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 2379, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 26, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 4, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 2379, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2379, 2024-09-21T01:32:36.233 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1.871957091, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000786867 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2379, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 2380 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 2379, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 2379, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 0, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 1498, 2024-09-21T01:32:36.234 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 878, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 2, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 0, 2024-09-21T01:32:36.235 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-09-21T01:32:36.236 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 98, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 10, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 5, 2024-09-21T01:32:36.237 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 5, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 0, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 2, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 0 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 0, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-09-21T01:32:36.238 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 2630, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 3114290, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:36.239 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 2630, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2630, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 3114290, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:36.240 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 2379, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1663494, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2379, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1663494, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:36.241 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 2379, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 2379, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2379, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2379, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.242 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:36.243 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55f193b75450": { 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:36.244 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "trackedop": { 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:36.245 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T01:32:36.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:35 smithi098 ceph-mon[34372]: pgmap v713: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-21T01:32:37.026 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T01:32:37.026 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T01:32:37.026 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-21T01:32:37.026 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-21T01:32:37.026 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-21T01:32:37.026 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:37.027 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:37.027 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T01:32:37.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T01:32:37.270 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 35, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 46, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 8904, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 63847, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 14, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.011164866, 2024-09-21T01:32:37.799 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.002553870, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.006263298, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000475187, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 44, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.019205179, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000436481 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 26, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000011932, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000458 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 736 2024-09-21T01:32:37.800 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 17, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 187, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 4236, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 405956, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 8, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.015354262, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.010381188, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002858999, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000303773, 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:37.801 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 185, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.026708692, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000144371 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000325, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000162 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 368880 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 787, 2024-09-21T01:32:37.802 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 830, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1217417, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 305492, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.121476615, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.062102287, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.066424729, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.003592553, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 830, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.051877360, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000062502 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.803 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1034, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000278193, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000269 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1060448, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 145984 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-09-21T01:32:37.804 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 56, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.001205223, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000021521 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000017670, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000008835 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.805 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 1, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 1, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 11, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2024-09-21T01:32:37.806 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 16, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 16, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 13, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 4, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 3, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 20, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 15, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 1, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 4, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:37.807 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 4, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 234, 2024-09-21T01:32:37.808 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:37.809 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2024-09-21T01:32:37.810 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 14, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 9, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 10, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:37.811 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 5, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 36, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_started": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_write_failed": 0 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-09-21T01:32:37.812 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 5, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 5, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 2, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2024-09-21T01:32:37.813 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 4195735, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 0, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 4, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.011542152, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.002885538 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 0 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 20, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 22, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 2, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 19, 2024-09-21T01:32:37.814 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 23, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 4, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 16, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 20, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 4, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 118320, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 324812800 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 1, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 1, 2024-09-21T01:32:37.815 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 9, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.818 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.819 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.820 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.821 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.822 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:37.823 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.824 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.825 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:37.826 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 1, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 2, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 1, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 1, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2024-09-21T01:32:37.827 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 307, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:37.828 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:37.829 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 237989, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 72, 2024-09-21T01:32:37.830 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 196, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T01:32:37.831 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 87517, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 393, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 40, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 11733, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 33, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 40, 2024-09-21T01:32:37.832 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 40, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.267993838, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.006699845 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 40, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 62 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 40, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 0, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 40, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-09-21T01:32:37.833 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 10, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 4, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 7, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 6, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 0, 2024-09-21T01:32:37.834 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 20, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-09-21T01:32:37.835 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 98, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-09-21T01:32:37.836 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 10, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 8, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 8, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 14, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 0, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 1 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 1, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-09-21T01:32:37.837 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 835, 2024-09-21T01:32:37.838 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1137959, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 835, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 835, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1137959, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:37.839 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 40, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 11733, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 40, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 11733, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.840 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 40, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 40, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 40, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 40, 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:37.841 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:37.842 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x5591e0fd5380": { 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 5, 2024-09-21T01:32:37.843 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1431, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 5, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 4, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1431, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "trackedop": { 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 33 2024-09-21T01:32:37.844 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:37.845 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T01:32:38.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:37 smithi089 ceph-mon[34486]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:32:38.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:37 smithi027 ceph-mon[30375]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:32:38.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:37 smithi098 ceph-mon[34372]: pgmap v714: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:32:38.569 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T01:32:38.569 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T01:32:38.569 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-21T01:32:38.569 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14 2024-09-21T01:32:38.570 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-21T01:32:38.570 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:38.570 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:38.570 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T01:32:38.570 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T01:32:38.814 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T01:32:39.411 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-09-21T01:32:39.412 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:39.412 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 758, 2024-09-21T01:32:39.412 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 758, 2024-09-21T01:32:39.412 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 256329, 2024-09-21T01:32:39.412 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 240221, 2024-09-21T01:32:39.412 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 6, 2024-09-21T01:32:39.413 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:39.413 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.113743466, 2024-09-21T01:32:39.413 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.047625949, 2024-09-21T01:32:39.413 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.026899801, 2024-09-21T01:32:39.413 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.029359665, 2024-09-21T01:32:39.413 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:39.414 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 757, 2024-09-21T01:32:39.414 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.052862966, 2024-09-21T01:32:39.414 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000069832 2024-09-21T01:32:39.414 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.414 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:39.414 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.415 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.415 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.415 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:39.415 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:39.415 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1380, 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1554, 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 352937, 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 808147, 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-09-21T01:32:39.416 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:39.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.141061861, 2024-09-21T01:32:39.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.069940456, 2024-09-21T01:32:39.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.026164512, 2024-09-21T01:32:39.417 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.035058333, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1552, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.080686876, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000051988 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.418 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 369568 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 255, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 247, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1036004, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 144885, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.047208364, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.021443710, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.037028313, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000075061, 2024-09-21T01:32:39.419 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 246, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.017594246, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000071521 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1034656, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 143920 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-09-21T01:32:39.420 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3642, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.106217997, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000029164 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-09-21T01:32:39.421 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 0, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2024-09-21T01:32:39.422 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 1, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 0, 2024-09-21T01:32:39.423 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:39.424 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:39.425 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2024-09-21T01:32:39.426 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 0, 2024-09-21T01:32:39.427 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_started": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "uninline_write_failed": 0 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2024-09-21T01:32:39.428 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 5, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 2, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 4194304, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 4195735, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 4195594, 2024-09-21T01:32:39.429 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 5 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 3, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 3, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 0, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 1, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 1, 2024-09-21T01:32:39.430 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 117252, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 316424192 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 0, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 0, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.433 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.434 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.435 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.436 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-09-21T01:32:39.437 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.438 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.439 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.440 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.441 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:39.442 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 1, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 2, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 1, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 2, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 3857, 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:39.443 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:39.444 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:39.445 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 197480, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 55, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-09-21T01:32:39.446 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 196, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 8672, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 13, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-09-21T01:32:39.447 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 2136, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 21, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 3, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 2136, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2136, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1.274970805, 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000596896 2024-09-21T01:32:39.448 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 2136, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 2137 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 2136, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 2136, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 0, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 1374, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 0, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 760, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 0, 2024-09-21T01:32:39.449 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 1, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:39.450 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-09-21T01:32:39.451 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 98, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 10, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 6, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 6, 2024-09-21T01:32:39.452 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 2, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 0 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:39.453 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 2389, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1414282, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 2389, 2024-09-21T01:32:39.454 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2389, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1414282, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:39.455 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 2136, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1436, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2136, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1436, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:39.456 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 2136, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 2136, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2136, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 2136, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.457 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T01:32:39.458 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55f4af2cf520": { 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-09-21T01:32:39.459 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "trackedop": { 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-09-21T01:32:39.460 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-09-21T01:32:40.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:39 smithi027 ceph-mon[30375]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:32:40.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:39 smithi089 ceph-mon[34486]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:40.290 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:40.290 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T01:32:40.290 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T01:32:40.318 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:39 smithi098 ceph-mon[34372]: pgmap v715: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:32:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 516, 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 576, 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 164404, 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 243831, 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 10, 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:41.079 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.071431909, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.031880969, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.033461715, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.005065785, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 575, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.041679799, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000072486 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1032, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000266898, 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000258 2024-09-21T01:32:41.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 305, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 297, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1052761, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 211539, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 11, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.052643910, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.022419734, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.042965816, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000188914, 2024-09-21T01:32:41.081 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 296, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.026081020, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000088111 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 26, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000003777, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000145 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1049680, 2024-09-21T01:32:41.082 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 151824 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 32, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 231, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 9055, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 385899, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.022713214, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.013728592, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.006209323, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000568454, 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:41.083 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 229, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.025932527, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000113242 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 33, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000020936, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000634 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 369056 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T01:32:41.084 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 48, 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001553184, 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000032358 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T01:32:41.085 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000021868, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000010934 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 6, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 5, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002482135, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000496427 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.086 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 1, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 16, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 16, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 13, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 4, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 2, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 2, 2024-09-21T01:32:41.087 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 3, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 25, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 21, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 4, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:41.088 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 4, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 234, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 5, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 5, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 14, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 9, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 10, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-09-21T01:32:41.090 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 5, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 36, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:41.091 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 5, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5, 2024-09-21T01:32:41.092 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195735, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-09-21T01:32:41.093 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.008099853, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002024963 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 20, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 22, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 2, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 19, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 23, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 4, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 16, 2024-09-21T01:32:41.094 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 20, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 4, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 2, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 5, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 3, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 118164, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 324808704 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 9, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 6, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 13, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 1, 2024-09-21T01:32:41.095 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001739429, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001739429 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.096 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000456919, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000228459 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000285787, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000142893 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.098 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.099 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.100 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.101 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.102 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.105 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-09-21T01:32:41.107 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 4, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 2, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 603, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:41.108 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:41.109 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T01:32:41.110 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 270117, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 82, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:41.111 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 196, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 88269, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 411, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T01:32:41.112 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 44, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 12239, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 33, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 44, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 44, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.253996650, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005772651 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 44, 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 69 2024-09-21T01:32:41.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 44, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 44, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 4, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 8, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T01:32:41.114 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T01:32:41.115 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T01:32:41.116 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 98, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 10, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 9, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 20, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 1 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-09-21T01:32:41.117 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:41.118 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 849, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1132217, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 849, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 849, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1132217, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.119 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 44, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 12239, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 44, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 12239, 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:41.120 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 44, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 44, 2024-09-21T01:32:41.121 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 44, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 44, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:41.122 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x556886a79380": { 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5, 2024-09-21T01:32:41.123 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1431, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1431, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 33 2024-09-21T01:32:41.124 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:41.125 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T01:32:41.948 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:41 smithi098 ceph-mon[34372]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:32:41.974 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T01:32:41.974 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T01:32:41.974 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-21T01:32:41.974 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-09-21T01:32:41.974 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-21T01:32:41.975 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:41.975 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:41.975 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T01:32:41.975 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T01:32:42.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:41 smithi027 ceph-mon[30375]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:32:42.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:41 smithi089 ceph-mon[34486]: pgmap v716: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-21T01:32:42.257 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:32:42.794 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T01:32:42.794 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:42.794 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2185, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2185, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 621862, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 694013, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.243864755, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.104365383, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.054116557, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.069979389, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2184, 2024-09-21T01:32:42.795 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.101636957, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000046537 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:42.796 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 251, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 243, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1034396, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 142714, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 3, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.045907053, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.020662712, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.034708694, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000069849, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 242, 2024-09-21T01:32:42.797 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.014933627, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000061709 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1033840, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 142016 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2, 2024-09-21T01:32:42.798 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 174, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 428, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 371182, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.013623362, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.010457840, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.001359163, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000028435, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 172, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.011518609, 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000066968 2024-09-21T01:32:42.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 370864 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.800 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3694, 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.114437453, 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000030979 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.801 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T01:32:42.802 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-09-21T01:32:42.803 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 1, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-09-21T01:32:42.804 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:42.805 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-09-21T01:32:42.806 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-09-21T01:32:42.807 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T01:32:42.808 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5, 2024-09-21T01:32:42.809 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195735, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 4195424, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.810 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 5 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 3, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 3, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 1, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 1, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 0, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 0, 2024-09-21T01:32:42.811 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 119596, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 316424192 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-09-21T01:32:42.812 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.815 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.817 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.819 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:42.821 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.822 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.823 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T01:32:42.824 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 2251, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:42.825 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:42.826 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:42.827 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 181303, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 46, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:42.828 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 196, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 8672, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 13, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T01:32:42.829 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 2184, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 21, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 3, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 2184, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2184, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.151984965, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000527465 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2184, 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 2185 2024-09-21T01:32:42.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 2184, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 2184, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 1418, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 764, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T01:32:42.831 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:42.832 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T01:32:42.833 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 98, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 10, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 4, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 4, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-09-21T01:32:42.834 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T01:32:42.835 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 2434, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1421804, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 2434, 2024-09-21T01:32:42.836 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2434, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1421804, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:42.837 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 2184, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 1436, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2184, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1436, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:42.838 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 2184, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 2184, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2184, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2184, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.839 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:42.840 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x563996d53380": { 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:42.841 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T01:32:42.842 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-09-21T01:32:42.843 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:42.843 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T01:32:43.664 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T01:32:43.664 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T01:32:43.664 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-21T01:32:43.664 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-21T01:32:43.664 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-21T01:32:43.665 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:43.665 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:43.665 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T01:32:43.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T01:32:43.908 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:32:44.170 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:43 smithi098 ceph-mon[34372]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:32:44.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:43 smithi027 ceph-mon[30375]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:32:44.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:43 smithi089 ceph-mon[34486]: pgmap v717: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-21T01:32:44.412 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T01:32:44.466 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:44.466 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 311, 2024-09-21T01:32:44.466 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 327, 2024-09-21T01:32:44.466 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1059741, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 238284, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 13, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.054868183, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.023910611, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.042283499, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000293021, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 326, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.026235585, 2024-09-21T01:32:44.467 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000080477 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 56, 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000010209, 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000182 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1055632, 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 157488 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.468 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 14, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 185, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3697, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 378716, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.014744000, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.010060479, 2024-09-21T01:32:44.469 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.002384305, 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000629854, 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 183, 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.025795748, 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000140960 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.470 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 372336 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 510, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 569, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 162294, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 196178, 2024-09-21T01:32:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.071797380, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.033820746, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.032538221, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.004939356, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 568, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.045155635, 2024-09-21T01:32:44.472 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000079499 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1035, 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000280104, 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000270 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-21T01:32:44.473 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T01:32:44.474 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 47, 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001791382, 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000038114 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-09-21T01:32:44.475 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-09-21T01:32:44.476 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-09-21T01:32:44.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-09-21T01:32:44.476 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000024355, 2024-09-21T01:32:44.476 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000012177 2024-09-21T01:32:44.476 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 5, 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 3, 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000537233, 2024-09-21T01:32:44.477 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000179077 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 2, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-09-21T01:32:44.478 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 16, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 16, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 13, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 4, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 1, 2024-09-21T01:32:44.479 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 1, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 3, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 26, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 21, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 4, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-09-21T01:32:44.480 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-09-21T01:32:44.481 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 4, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 234, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T01:32:44.482 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 1, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 1, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.483 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 14, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 9, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 10, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-09-21T01:32:44.484 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 5, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 36, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_started": 0, 2024-09-21T01:32:44.485 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_succeeded": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "uninline_write_failed": 0 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 3, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 3, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-09-21T01:32:44.486 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195283, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004096899, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002048449 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.487 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 20, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 22, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 2, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 19, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 23, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 4, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 16, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 20, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 4, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 1, 2024-09-21T01:32:44.488 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 1, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 118516, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 324804608 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 5, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 5, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 8, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-09-21T01:32:44.489 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000537233, 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000179077 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.490 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-09-21T01:32:44.491 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.493 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.494 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.494 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.494 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-09-21T01:32:44.494 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.494 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-09-21T01:32:44.495 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-09-21T01:32:44.496 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-09-21T01:32:44.497 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-09-21T01:32:44.498 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.499 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.500 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 1, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 0, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-09-21T01:32:44.501 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 2, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 2, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 582, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:44.502 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:44.503 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 246359, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 80, 2024-09-21T01:32:44.504 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 9648, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 196, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 87845, 2024-09-21T01:32:44.505 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 396, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 37, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 11191, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 29, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 37, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 37, 2024-09-21T01:32:44.506 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.251996674, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.006810720 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 37, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 59 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 37, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 37, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-09-21T01:32:44.507 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 2, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 6, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-09-21T01:32:44.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-09-21T01:32:44.509 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 98, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 10, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 8, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 8, 2024-09-21T01:32:44.510 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 14, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 1 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T01:32:44.511 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 831, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1133122, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 831, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:44.512 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 831, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1133122, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:44.513 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 37, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 11191, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 37, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 11191, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:44.514 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 37, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 37, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 37, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 37, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.516 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x5598daacd450": { 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 979, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:44.517 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2, 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 979, 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "trackedop": { 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 33 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:44.518 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=14 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T01:32:45.201 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T01:32:45.201 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T01:32:45.201 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T01:32:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 241, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 233, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 999840, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 143072, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 0, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.048826704, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.021678872, 2024-09-21T01:32:46.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.037402574, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 233, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017196149, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000073803 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 999840, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 143072 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T01:32:46.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 177, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 160, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 111440, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 3, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.014832155, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.011934593, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000690354, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 175, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.010948408, 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000062562 2024-09-21T01:32:46.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 111440 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 0, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 0, 2024-09-21T01:32:46.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 0, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 2, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.000265153, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.000199145, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:46.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-09-21T01:32:46.012 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T01:32:46.013 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T01:32:46.014 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 148967, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 29, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-09-21T01:32:46.015 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 968, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 2, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 238, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 963287, 2024-09-21T01:32:46.016 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 238, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 238, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 963287, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-21T01:32:46.017 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-21T01:32:46.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:45 smithi098 ceph-mon[34372]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-21T01:32:46.169 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:45 smithi089 ceph-mon[34486]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-21T01:32:46.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:45 smithi027 ceph-mon[30375]: pgmap v718: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-09-21T01:32:46.677 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-21T01:32:46.678 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-21T01:32:46.678 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-21T01:32:46.678 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-21T01:32:46.678 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T01:32:46.678 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T01:32:46.678 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T01:32:46.678 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T01:32:46.694 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T01:32:46.695 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:32:46.695 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T01:32:46.739 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-09-21T01:32:46.739 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T01:32:46.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T01:32:47.250 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:47.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 -- 172.21.15.27:0/151523589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 msgr2=0x7f3728077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:47.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 --2- 172.21.15.27:0/151523589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 0x7f3728077d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3718009900 tx=0x7f371802a410 comp rx=0 tx=0).stop 2024-09-21T01:32:47.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 -- 172.21.15.27:0/151523589 shutdown_connections 2024-09-21T01:32:47.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 --2- 172.21.15.27:0/151523589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3728114740 0x7f3728116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:47.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 --2- 172.21.15.27:0/151523589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f3728114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:47.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 --2- 172.21.15.27:0/151523589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 0x7f3728077d50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:47.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 -- 172.21.15.27:0/151523589 >> 172.21.15.27:0/151523589 conn(0x7f3728101810 msgr2=0x7f3728103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:47.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 -- 172.21.15.27:0/151523589 shutdown_connections 2024-09-21T01:32:47.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.937+0000 7f372f400640 1 -- 172.21.15.27:0/151523589 wait complete. 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.938+0000 7f372f400640 1 Processor -- start 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 -- start start 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 0x7f37281a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f37281a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3728114740 0x7f37281a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f372811a100 con 0x7f3728077940 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3728119f80 con 0x7f3728114740 2024-09-21T01:32:47.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.939+0000 7f372f400640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f372811a280 con 0x7f3728078290 2024-09-21T01:32:47.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f37281a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:47.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f372ca00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 0x7f37281a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f372d400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3728114740 0x7f37281a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f37281a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33726/0 (socket says 172.21.15.27:33726) 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 -- 172.21.15.27:0/3143107767 learned_addr learned my addr 172.21.15.27:0/3143107767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 -- 172.21.15.27:0/3143107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3728114740 msgr2=0x7f37281a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3728114740 0x7f37281a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 -- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 msgr2=0x7f37281a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 0x7f37281a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 -- 172.21.15.27:0/3143107767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281a6a00 con 0x7f3728078290 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.940+0000 7f371fe00640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f37281a1030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f371000d740 tx=0x7f371000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.941+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710017070 con 0x7f3728078290 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.941+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f371000de50 con 0x7f3728078290 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.941+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37281a6cf0 con 0x7f3728078290 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.941+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371000f620 con 0x7f3728078290 2024-09-21T01:32:47.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.941+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281ad450 con 0x7f3728078290 2024-09-21T01:32:47.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.942+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728077d50 con 0x7f3728078290 2024-09-21T01:32:47.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.944+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f371001a070 con 0x7f3728078290 2024-09-21T01:32:47.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.947+0000 7f3726000640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f370c0b6700 0x7f370c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:47.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.947+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7f37100e7430 con 0x7f3728078290 2024-09-21T01:32:47.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.947+0000 7f372ca00640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f370c0b6700 0x7f370c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:47.948 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.948+0000 7f372ca00640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f370c0b6700 0x7f370c0b8c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3718005ab0 tx=0x7f3718005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:47.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:47.950+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37100b1930 con 0x7f3728078290 2024-09-21T01:32:48.034 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:47 smithi027 ceph-mon[30375]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:32:48.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.202+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37281a1bc0 con 0x7f3728078290 2024-09-21T01:32:48.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.203+0000 7f3726000640 1 -- 172.21.15.27:0/3143107767 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v98) ==== 81+0+885 (secure 0 0 0) 0x7f37100b6870 con 0x7f3728078290 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/525441395 2024-09-22T01:11:40.306879+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3673025816 2024-09-22T01:10:00.491967+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3673025816 2024-09-22T01:10:00.491967+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3489401183 2024-09-22T01:10:00.491967+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/2219512661 2024-09-22T01:10:00.491967+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1438694632 2024-09-22T01:10:00.491967+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/3049262375 2024-09-22T01:10:32.417709+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/3134270458 2024-09-22T01:10:32.417709+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/3049262375 2024-09-22T01:10:32.417709+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/365392204 2024-09-22T01:10:32.417709+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1444171027 2024-09-22T01:11:40.306879+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/492430082 2024-09-22T01:10:32.417709+0000 2024-09-21T01:32:48.204 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6800/1523735126 2024-09-22T01:11:40.306879+0000 2024-09-21T01:32:48.205 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:0/1528176571 2024-09-22T01:11:40.306879+0000 2024-09-21T01:32:48.205 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6801/1523735126 2024-09-22T01:11:40.306879+0000 2024-09-21T01:32:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:listed 15 entries 2024-09-21T01:32:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f370c0b6700 msgr2=0x7f370c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f370c0b6700 0x7f370c0b8c10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3718005ab0 tx=0x7f3718005a20 comp rx=0 tx=0).stop 2024-09-21T01:32:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 msgr2=0x7f37281a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f37281a1030 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f371000d740 tx=0x7f371000dc00 comp rx=0 tx=0).stop 2024-09-21T01:32:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 shutdown_connections 2024-09-21T01:32:48.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f370c0b6700 0x7f370c0b8c10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3728114740 0x7f37281a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3728078290 0x7f37281a1030 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 --2- 172.21.15.27:0/3143107767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3728077940 0x7f37281a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 >> 172.21.15.27:0/3143107767 conn(0x7f3728101810 msgr2=0x7f3728116810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 shutdown_connections 2024-09-21T01:32:48.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:48.205+0000 7f372f400640 1 -- 172.21.15.27:0/3143107767 wait complete. 2024-09-21T01:32:48.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:47 smithi089 ceph-mon[34486]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:32:48.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:47 smithi098 ceph-mon[34372]: pgmap v719: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-09-21T01:32:49.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:48 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/3143107767' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:32:49.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:48 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/3143107767' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:32:49.267 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T01:32:49.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:32:49.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T01:32:49.299 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:48 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/3143107767' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T01:32:49.352 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:32:49.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:32:49.352 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T01:32:49.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:32:49.406 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-09-21T01:32:49.442 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T01:32:49.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:32:49.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T01:32:49.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T01:32:49.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-09-21T01:32:49.894 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T01:32:49.913 INFO:tasks.cephadm:Teardown begin 2024-09-21T01:32:49.913 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T01:32:49.914 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:32:50.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:49 smithi027 ceph-mon[30375]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:32:50.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:49 smithi089 ceph-mon[34486]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:32:50.302 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:50.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:49 smithi098 ceph-mon[34372]: pgmap v720: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 -- 172.21.15.27:0/301528448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f8110040 msgr2=0x7fe4f8112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 --2- 172.21.15.27:0/301528448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f8110040 0x7fe4f8112550 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009ca0 tx=0x7fe4f002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 -- 172.21.15.27:0/301528448 shutdown_connections 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 --2- 172.21.15.27:0/301528448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f8110040 0x7fe4f8112550 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 --2- 172.21.15.27:0/301528448 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f806c0e0 0x7fe4f806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 --2- 172.21.15.27:0/301528448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f81046a0 0x7fe4f806bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 -- 172.21.15.27:0/301528448 >> 172.21.15.27:0/301528448 conn(0x7fe4f80799a0 msgr2=0x7fe4f8079db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 -- 172.21.15.27:0/301528448 shutdown_connections 2024-09-21T01:32:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.007+0000 7fe4fd000640 1 -- 172.21.15.27:0/301528448 wait complete. 2024-09-21T01:32:51.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 Processor -- start 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 -- start start 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f806c0e0 0x7fe4f819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f81046a0 0x7fe4f819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f8110040 0x7fe4f81a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8115cc0 con 0x7fe4f81046a0 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8115b40 con 0x7fe4f806c0e0 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.009+0000 7fe4fd000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8115e40 con 0x7fe4f8110040 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f81046a0 0x7fe4f819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f8110040 0x7fe4f81a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f5400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f81046a0 0x7fe4f819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34674/0 (socket says 172.21.15.27:34674) 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f5400640 1 -- 172.21.15.27:0/2891378440 learned_addr learned my addr 172.21.15.27:0/2891378440 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f5e00640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f806c0e0 0x7fe4f819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 -- 172.21.15.27:0/2891378440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f806c0e0 msgr2=0x7fe4f819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f806c0e0 0x7fe4f819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 -- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f81046a0 msgr2=0x7fe4f819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f81046a0 0x7fe4f819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 -- 172.21.15.27:0/2891378440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f81a2580 con 0x7fe4f8110040 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.010+0000 7fe4f6800640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f8110040 0x7fe4f81a1e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4f002ade0 tx=0x7fe4f0031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.011+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003b70 con 0x7fe4f8110040 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.011+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0031910 con 0x7fe4f8110040 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.011+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f002f950 con 0x7fe4f8110040 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.011+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f81a2810 con 0x7fe4f8110040 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.011+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81a8fd0 con 0x7fe4f8110040 2024-09-21T01:32:51.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.012+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f8102c60 con 0x7fe4f8110040 2024-09-21T01:32:51.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.014+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fe4f002faf0 con 0x7fe4f8110040 2024-09-21T01:32:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.016+0000 7fe4eea00640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe4dc0b6700 0x7fe4dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.016+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7fe4f0108b10 con 0x7fe4f8110040 2024-09-21T01:32:51.016 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.016+0000 7fe4f5e00640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe4dc0b6700 0x7fe4dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:51.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.017+0000 7fe4f5e00640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe4dc0b6700 0x7fe4dc0b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009bd0 tx=0x7fe4e8005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:51.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.020+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4f00cec30 con 0x7fe4f8110040 2024-09-21T01:32:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.259+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4f81a9200 con 0x7fe4f8110040 2024-09-21T01:32:51.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.260+0000 7fe4eea00640 1 -- 172.21.15.27:0/2891378440 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7fe4f0003d10 con 0x7fe4f8110040 2024-09-21T01:32:51.261 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:32:51.261 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:32:51.261 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:32:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe4dc0b6700 msgr2=0x7fe4dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe4dc0b6700 0x7fe4dc0b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009bd0 tx=0x7fe4e8005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f8110040 msgr2=0x7fe4f81a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f8110040 0x7fe4f81a1e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe4f002ade0 tx=0x7fe4f0031660 comp rx=0 tx=0).stop 2024-09-21T01:32:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 shutdown_connections 2024-09-21T01:32:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fe4dc0b6700 0x7fe4dc0b8c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4f8110040 0x7fe4f81a1e50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4f81046a0 0x7fe4f819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 --2- 172.21.15.27:0/2891378440 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4f806c0e0 0x7fe4f819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.262+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 >> 172.21.15.27:0/2891378440 conn(0x7fe4f80799a0 msgr2=0x7fe4f81014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.263+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 shutdown_connections 2024-09-21T01:32:51.264 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:51.263+0000 7fe4fd000640 1 -- 172.21.15.27:0/2891378440 wait complete. 2024-09-21T01:32:52.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T01:32:52.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:51 smithi089 ceph-mon[34486]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T01:32:52.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:51 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2891378440' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:32:52.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:51 smithi098 ceph-mon[34372]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T01:32:52.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:51 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2891378440' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:32:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:51 smithi027 ceph-mon[30375]: pgmap v721: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-09-21T01:32:52.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:51 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2891378440' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:32:52.434 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:53.132 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.131+0000 7eff12e00640 1 -- 172.21.15.27:0/3978754009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c102d30 msgr2=0x7eff0c10fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.131+0000 7eff12e00640 1 --2- 172.21.15.27:0/3978754009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c102d30 0x7eff0c10fa50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efef8009900 tx=0x7efef802a480 comp rx=0 tx=0).stop 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 -- 172.21.15.27:0/3978754009 shutdown_connections 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 --2- 172.21.15.27:0/3978754009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c10ff90 0x7eff0c1123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 --2- 172.21.15.27:0/3978754009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c102d30 0x7eff0c10fa50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 --2- 172.21.15.27:0/3978754009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff0c1023e0 0x7eff0c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 -- 172.21.15.27:0/3978754009 >> 172.21.15.27:0/3978754009 conn(0x7eff0c0fe2c0 msgr2=0x7eff0c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 -- 172.21.15.27:0/3978754009 shutdown_connections 2024-09-21T01:32:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.132+0000 7eff12e00640 1 -- 172.21.15.27:0/3978754009 wait complete. 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 Processor -- start 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 -- start start 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff0c1023e0 0x7eff0c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c102d30 0x7eff0c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c10ff90 0x7eff0c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c115ad0 con 0x7eff0c10ff90 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c115950 con 0x7eff0c1023e0 2024-09-21T01:32:53.134 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.133+0000 7eff12e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c115c50 con 0x7eff0c102d30 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff11000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c102d30 0x7eff0c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff12400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c10ff90 0x7eff0c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff11000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c102d30 0x7eff0c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33760/0 (socket says 172.21.15.27:33760) 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff11000640 1 -- 172.21.15.27:0/2993623321 learned_addr learned my addr 172.21.15.27:0/2993623321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff11a00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff0c1023e0 0x7eff0c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff12400640 1 -- 172.21.15.27:0/2993623321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c102d30 msgr2=0x7eff0c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff12400640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c102d30 0x7eff0c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff12400640 1 -- 172.21.15.27:0/2993623321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff0c1023e0 msgr2=0x7eff0c19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff12400640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff0c1023e0 0x7eff0c19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.134+0000 7eff12400640 1 -- 172.21.15.27:0/2993623321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c1a2410 con 0x7eff0c10ff90 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.135+0000 7eff12400640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c10ff90 0x7eff0c1a1ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7eff04009fd0 tx=0x7eff0400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:53.136 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.135+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0400ce20 con 0x7eff0c10ff90 2024-09-21T01:32:53.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.135+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff04005020 con 0x7eff0c10ff90 2024-09-21T01:32:53.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.135+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04010690 con 0x7eff0c10ff90 2024-09-21T01:32:53.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.135+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c1a2700 con 0x7eff0c10ff90 2024-09-21T01:32:53.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.135+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1a8e60 con 0x7eff0c10ff90 2024-09-21T01:32:53.137 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.136+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c1027f0 con 0x7eff0c10ff90 2024-09-21T01:32:53.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.139+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7eff040054d0 con 0x7eff0c10ff90 2024-09-21T01:32:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.140+0000 7eff0a000640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efef40b6700 0x7efef40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.141+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7eff040e8160 con 0x7eff0c10ff90 2024-09-21T01:32:53.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.141+0000 7eff11a00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efef40b6700 0x7efef40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:53.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.142+0000 7eff11a00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efef40b6700 0x7efef40b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff00009bd0 tx=0x7eff00005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:53.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.143+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff040ae280 con 0x7eff0c10ff90 2024-09-21T01:32:53.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.383+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff0c19d5d0 con 0x7eff0c10ff90 2024-09-21T01:32:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.384+0000 7eff0a000640 1 -- 172.21.15.27:0/2993623321 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11960 (secure 0 0 0) 0x7eff0c19d5d0 con 0x7eff0c10ff90 2024-09-21T01:32:53.385 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:32:53.385 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"btime":"2024-09-21T01:22:16:639138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6841/2646952373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6840","nonce":2646952373},{"type":"v1","addr":"172.21.15.27:6841","nonce":2646952373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24661,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2535514726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2535514726},{"type":"v1","addr":"172.21.15.98:6833","nonce":2535514726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T01:19:07.935504+0000","modified":"2024-09-21T01:22:15.795685+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24776,"mds_1":24752,"mds_2":24734,"mds_3":24730,"mds_4":24718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6833/39744273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":39744273},{"type":"v1","addr":"172.21.15.89:6833","nonce":39744273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24686":{"gid":24686,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/2864798916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2864798916},{"type":"v1","addr":"172.21.15.27:6835","nonce":2864798916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24695":{"gid":24695,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6837/392681445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":392681445},{"type":"v1","addr":"172.21.15.27:6837","nonce":392681445}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/1764678372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1764678372},{"type":"v1","addr":"172.21.15.98:6835","nonce":1764678372}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6835/3850699909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3850699909},{"type":"v1","addr":"172.21.15.89:6835","nonce":3850699909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24718":{"gid":24718,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":23,"addr":"172.21.15.98:6837/1081631404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1081631404},{"type":"v1","addr":"172.21.15.98:6837","nonce":1081631404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.98:6839/3799755243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":3799755243},{"type":"v1","addr":"172.21.15.98:6839","nonce":3799755243}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24734":{"gid":24734,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":22,"addr":"172.21.15.89:6837/3556269004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":3556269004},{"type":"v1","addr":"172.21.15.89:6837","nonce":3556269004}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24752":{"gid":24752,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6839/3115497172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3115497172},{"type":"v1","addr":"172.21.15.27:6839","nonce":3115497172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24776":{"gid":24776,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.89:6839/4156411896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":4156411896},{"type":"v1","addr":"172.21.15.89:6839","nonce":4156411896}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24776,"qdb_cluster":[24776,24752,24734,24730,24718]},"id":1}]} 2024-09-21T01:32:53.385 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-09-21T01:32:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.387+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efef40b6700 msgr2=0x7efef40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.387+0000 7eff12e00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efef40b6700 0x7efef40b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff00009bd0 tx=0x7eff00005f90 comp rx=0 tx=0).stop 2024-09-21T01:32:53.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.387+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c10ff90 msgr2=0x7eff0c1a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.387+0000 7eff12e00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c10ff90 0x7eff0c1a1ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7eff04009fd0 tx=0x7eff0400ef10 comp rx=0 tx=0).stop 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 shutdown_connections 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7efef40b6700 0x7efef40b8c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff0c10ff90 0x7eff0c1a1ce0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff0c102d30 0x7eff0c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 --2- 172.21.15.27:0/2993623321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7eff0c1023e0 0x7eff0c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 >> 172.21.15.27:0/2993623321 conn(0x7eff0c0fe2c0 msgr2=0x7eff0c1120e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 shutdown_connections 2024-09-21T01:32:53.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:53.388+0000 7eff12e00640 1 -- 172.21.15.27:0/2993623321 wait complete. 2024-09-21T01:32:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:53 smithi027 ceph-mon[30375]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:32:54.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:53 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/2993623321' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:32:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:53 smithi089 ceph-mon[34486]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:32:54.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:53 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/2993623321' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:32:54.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T01:32:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:53 smithi098 ceph-mon[34372]: pgmap v722: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-21T01:32:54.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:53 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/2993623321' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T01:32:54.713 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:55.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 -- 172.21.15.27:0/3787346157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078290 msgr2=0x7fbd48114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:55.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/3787346157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078290 0x7fbd48114200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009900 tx=0x7fbd3002a410 comp rx=0 tx=0).stop 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 -- 172.21.15.27:0/3787346157 shutdown_connections 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/3787346157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48114740 0x7fbd48116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/3787346157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48078290 0x7fbd48114200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/3787346157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd48077940 0x7fbd48077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 -- 172.21.15.27:0/3787346157 >> 172.21.15.27:0/3787346157 conn(0x7fbd481017d0 msgr2=0x7fbd48103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.373+0000 7fbd4dc00640 1 -- 172.21.15.27:0/3787346157 shutdown_connections 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.374+0000 7fbd4dc00640 1 -- 172.21.15.27:0/3787346157 wait complete. 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.375+0000 7fbd4dc00640 1 Processor -- start 2024-09-21T01:32:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.375+0000 7fbd4dc00640 1 -- start start 2024-09-21T01:32:55.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd4dc00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 0x7fbd481a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:55.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd4dc00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd48078290 0x7fbd481a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:55.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd4dc00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48114740 0x7fbd481a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:55.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd4dc00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811a0e0 con 0x7fbd48114740 2024-09-21T01:32:55.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd4dc00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48119f60 con 0x7fbd48078290 2024-09-21T01:32:55.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd4dc00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4811a260 con 0x7fbd48077940 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd47400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48114740 0x7fbd481a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd46a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 0x7fbd481a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd46a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 0x7fbd481a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.27:33786/0 (socket says 172.21.15.27:33786) 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd46a00640 1 -- 172.21.15.27:0/384965050 learned_addr learned my addr 172.21.15.27:0/384965050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd47400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48114740 0x7fbd481a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34708/0 (socket says 172.21.15.27:34708) 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.376+0000 7fbd3fe00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd48078290 0x7fbd481a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd46a00640 1 -- 172.21.15.27:0/384965050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd48078290 msgr2=0x7fbd481a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd46a00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd48078290 0x7fbd481a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd46a00640 1 -- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48114740 msgr2=0x7fbd481a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd46a00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48114740 0x7fbd481a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd46a00640 1 -- 172.21.15.27:0/384965050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd481a6a00 con 0x7fbd48077940 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd46a00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 0x7fbd481a0a90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800ea60 tx=0x7fbd3800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:55.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3800cde0 con 0x7fbd48077940 2024-09-21T01:32:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481a6cf0 con 0x7fbd48077940 2024-09-21T01:32:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd380035f0 con 0x7fbd48077940 2024-09-21T01:32:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.377+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38018690 con 0x7fbd48077940 2024-09-21T01:32:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.378+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481ad450 con 0x7fbd48077940 2024-09-21T01:32:55.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.378+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd48077d50 con 0x7fbd48077940 2024-09-21T01:32:55.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.381+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fbd38003a10 con 0x7fbd48077940 2024-09-21T01:32:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.383+0000 7fbd3f400640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbd2c0b6700 0x7fbd2c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.383+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7fbd380e7e30 con 0x7fbd48077940 2024-09-21T01:32:55.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.383+0000 7fbd3fe00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbd2c0b6700 0x7fbd2c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:55.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.384+0000 7fbd3fe00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbd2c0b6700 0x7fbd2c0b8c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd3002aa00 tx=0x7fbd30005a20 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:55.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.384+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd380adfe0 con 0x7fbd48077940 2024-09-21T01:32:55.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.584+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd481a1bc0 con 0x7fbd48077940 2024-09-21T01:32:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.586+0000 7fbd3f400640 1 -- 172.21.15.27:0/384965050 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+25331 (secure 0 0 0) 0x7fbd380b23c0 con 0x7fbd48077940 2024-09-21T01:32:55.590 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-09-21T01:32:55.590 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":98,"fsid":"fe42ec90-77b5-11ef-baf5-efdc52797490","created":"2024-09-21T01:09:34.737222+0000","modified":"2024-09-21T01:32:21.575057+0000","last_up_change":"2024-09-21T01:18:07.625098+0000","last_in_change":"2024-09-21T01:17:43.067535+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T01:13:52.790954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T01:19:02.303884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T01:19:04.941508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T01:19:14.791527+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ac7acdbc-3b48-4cdc-b04e-46d6930e274d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6803","nonce":3167750643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6805","nonce":3167750643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6809","nonce":3167750643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3167750643},{"type":"v1","addr":"172.21.15.27:6807","nonce":3167750643}]},"public_addr":"172.21.15.27:6803/3167750643","cluster_addr":"172.21.15.27:6805/3167750643","heartbeat_back_addr":"172.21.15.27:6809/3167750643","heartbeat_front_addr":"172.21.15.27:6807/3167750643","state":["exists","up"]},{"osd":1,"uuid":"5714e04c-89ef-47b5-9d28-8cc23530a45e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6801","nonce":1010396309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6803","nonce":1010396309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6807","nonce":1010396309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":1010396309},{"type":"v1","addr":"172.21.15.89:6805","nonce":1010396309}]},"public_addr":"172.21.15.89:6801/1010396309","cluster_addr":"172.21.15.89:6803/1010396309","heartbeat_back_addr":"172.21.15.89:6807/1010396309","heartbeat_front_addr":"172.21.15.89:6805/1010396309","state":["exists","up"]},{"osd":2,"uuid":"3493a644-896e-4c12-bcf2-62a3aaa3f7d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6801","nonce":2706735365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6803","nonce":2706735365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6807","nonce":2706735365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":2706735365},{"type":"v1","addr":"172.21.15.98:6805","nonce":2706735365}]},"public_addr":"172.21.15.98:6801/2706735365","cluster_addr":"172.21.15.98:6803/2706735365","heartbeat_back_addr":"172.21.15.98:6807/2706735365","heartbeat_front_addr":"172.21.15.98:6805/2706735365","state":["exists","up"]},{"osd":3,"uuid":"ae443ec8-2955-493e-a422-395e3948b860","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6811","nonce":4289720168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6813","nonce":4289720168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6817","nonce":4289720168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":4289720168},{"type":"v1","addr":"172.21.15.27:6815","nonce":4289720168}]},"public_addr":"172.21.15.27:6811/4289720168","cluster_addr":"172.21.15.27:6813/4289720168","heartbeat_back_addr":"172.21.15.27:6817/4289720168","heartbeat_front_addr":"172.21.15.27:6815/4289720168","state":["exists","up"]},{"osd":4,"uuid":"00c2670f-c063-4152-8f27-c437a83f1e03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6809","nonce":2496831965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6811","nonce":2496831965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6815","nonce":2496831965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":2496831965},{"type":"v1","addr":"172.21.15.89:6813","nonce":2496831965}]},"public_addr":"172.21.15.89:6809/2496831965","cluster_addr":"172.21.15.89:6811/2496831965","heartbeat_back_addr":"172.21.15.89:6815/2496831965","heartbeat_front_addr":"172.21.15.89:6813/2496831965","state":["exists","up"]},{"osd":5,"uuid":"a8fa79ae-691d-44f3-b727-db027e158d64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6809","nonce":3161787475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6811","nonce":3161787475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6815","nonce":3161787475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":3161787475},{"type":"v1","addr":"172.21.15.98:6813","nonce":3161787475}]},"public_addr":"172.21.15.98:6809/3161787475","cluster_addr":"172.21.15.98:6811/3161787475","heartbeat_back_addr":"172.21.15.98:6815/3161787475","heartbeat_front_addr":"172.21.15.98:6813/3161787475","state":["exists","up"]},{"osd":6,"uuid":"17a579ce-a646-47ff-abcc-9d967c03d62c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6819","nonce":1151323802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6821","nonce":1151323802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6825","nonce":1151323802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":1151323802},{"type":"v1","addr":"172.21.15.27:6823","nonce":1151323802}]},"public_addr":"172.21.15.27:6819/1151323802","cluster_addr":"172.21.15.27:6821/1151323802","heartbeat_back_addr":"172.21.15.27:6825/1151323802","heartbeat_front_addr":"172.21.15.27:6823/1151323802","state":["exists","up"]},{"osd":7,"uuid":"3ce20c01-8543-4ca7-8986-e2cee9ae1f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6817","nonce":2410726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6819","nonce":2410726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6823","nonce":2410726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2410726915},{"type":"v1","addr":"172.21.15.89:6821","nonce":2410726915}]},"public_addr":"172.21.15.89:6817/2410726915","cluster_addr":"172.21.15.89:6819/2410726915","heartbeat_back_addr":"172.21.15.89:6823/2410726915","heartbeat_front_addr":"172.21.15.89:6821/2410726915","state":["exists","up"]},{"osd":8,"uuid":"86419495-74dd-48c5-a17f-c4c4c7411658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6817","nonce":1001086058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6819","nonce":1001086058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6823","nonce":1001086058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1001086058},{"type":"v1","addr":"172.21.15.98:6821","nonce":1001086058}]},"public_addr":"172.21.15.98:6817/1001086058","cluster_addr":"172.21.15.98:6819/1001086058","heartbeat_back_addr":"172.21.15.98:6823/1001086058","heartbeat_front_addr":"172.21.15.98:6821/1001086058","state":["exists","up"]},{"osd":9,"uuid":"7d3a6679-2483-45b0-99b5-55c8668cb2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6827","nonce":1133805797}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6829","nonce":1133805797}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6833","nonce":1133805797}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1133805797},{"type":"v1","addr":"172.21.15.27:6831","nonce":1133805797}]},"public_addr":"172.21.15.27:6827/1133805797","cluster_addr":"172.21.15.27:6829/1133805797","heartbeat_back_addr":"172.21.15.27:6833/1133805797","heartbeat_front_addr":"172.21.15.27:6831/1133805797","state":["exists","up"]},{"osd":10,"uuid":"1978e43b-cf1a-4a0c-a1bf-11a8ce290970","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6825","nonce":4138240961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6827","nonce":4138240961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6831","nonce":4138240961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":4138240961},{"type":"v1","addr":"172.21.15.89:6829","nonce":4138240961}]},"public_addr":"172.21.15.89:6825/4138240961","cluster_addr":"172.21.15.89:6827/4138240961","heartbeat_back_addr":"172.21.15.89:6831/4138240961","heartbeat_front_addr":"172.21.15.89:6829/4138240961","state":["exists","up"]},{"osd":11,"uuid":"0eb4e911-7e6f-4cf0-b60c-983aed5f5b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6825","nonce":165111690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6827","nonce":165111690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6831","nonce":165111690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":165111690},{"type":"v1","addr":"172.21.15.98:6829","nonce":165111690}]},"public_addr":"172.21.15.98:6825/165111690","cluster_addr":"172.21.15.98:6827/165111690","heartbeat_back_addr":"172.21.15.98:6831/165111690","heartbeat_front_addr":"172.21.15.98:6829/165111690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:12:57.150718+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:24.469405+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:13:49.266743+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:20.788300+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:14:47.687554+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:14.110948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:15:46.238616+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:13.945120+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:16:40.898132+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:11.300193+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:17:39.253253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T01:18:05.053951+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.2","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:0/525441395":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6800/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6801/3673025816":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/3489401183":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/2219512661":"2024-09-22T01:10:00.491967+0000","172.21.15.27:0/1438694632":"2024-09-22T01:10:00.491967+0000","172.21.15.27:6800/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/3134270458":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6801/3049262375":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/365392204":"2024-09-22T01:10:32.417709+0000","172.21.15.27:0/1444171027":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/492430082":"2024-09-22T01:10:32.417709+0000","172.21.15.27:6800/1523735126":"2024-09-22T01:11:40.306879+0000","172.21.15.27:0/1528176571":"2024-09-22T01:11:40.306879+0000","172.21.15.27:6801/1523735126":"2024-09-22T01:11:40.306879+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T01:32:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.588+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbd2c0b6700 msgr2=0x7fbd2c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbd2c0b6700 0x7fbd2c0b8c10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd3002aa00 tx=0x7fbd30005a20 comp rx=0 tx=0).stop 2024-09-21T01:32:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 msgr2=0x7fbd481a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 0x7fbd481a0a90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800ea60 tx=0x7fbd3800ef20 comp rx=0 tx=0).stop 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 shutdown_connections 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7fbd2c0b6700 0x7fbd2c0b8c10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48114740 0x7fbd481a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd48078290 0x7fbd481a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 --2- 172.21.15.27:0/384965050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd48077940 0x7fbd481a0a90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 >> 172.21.15.27:0/384965050 conn(0x7fbd481017d0 msgr2=0x7fbd48116860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 shutdown_connections 2024-09-21T01:32:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:55.589+0000 7fbd4dc00640 1 -- 172.21.15.27:0/384965050 wait complete. 2024-09-21T01:32:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:55 smithi089 ceph-mon[34486]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T01:32:56.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:55 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/384965050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:32:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:55 smithi027 ceph-mon[30375]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T01:32:56.297 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:55 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/384965050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:32:56.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T01:32:56.397 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:55 smithi098 ceph-mon[34372]: pgmap v723: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T01:32:56.398 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:55 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/384965050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T01:32:56.722 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:57.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 -- 172.21.15.27:0/153595611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41039b0 msgr2=0x7f02b4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:57.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 --2- 172.21.15.27:0/153595611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41039b0 0x7f02b4103dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009900 tx=0x7f02a402a3f0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 -- 172.21.15.27:0/153595611 shutdown_connections 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 --2- 172.21.15.27:0/153595611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b4109c40 0x7f02b4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 --2- 172.21.15.27:0/153595611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41092f0 0x7f02b4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 --2- 172.21.15.27:0/153595611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41039b0 0x7f02b4103dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 -- 172.21.15.27:0/153595611 >> 172.21.15.27:0/153595611 conn(0x7f02b40ff440 msgr2=0x7f02b4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 -- 172.21.15.27:0/153595611 shutdown_connections 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.379+0000 7f02b9000640 1 -- 172.21.15.27:0/153595611 wait complete. 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.380+0000 7f02b9000640 1 Processor -- start 2024-09-21T01:32:57.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.380+0000 7f02b9000640 1 -- start start 2024-09-21T01:32:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b9000640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41039b0 0x7f02b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b9000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41092f0 0x7f02b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b4109c40 0x7f02b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b9000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4113900 con 0x7f02b4109c40 2024-09-21T01:32:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b9000640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4113780 con 0x7f02b41039b0 2024-09-21T01:32:57.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b9000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4113a80 con 0x7f02b41092f0 2024-09-21T01:32:57.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41039b0 0x7f02b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:57.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.381+0000 7f02b2a00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41092f0 0x7f02b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:57.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b3400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41039b0 0x7f02b4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:54954/0 (socket says 172.21.15.27:54954) 2024-09-21T01:32:57.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b3400640 1 -- 172.21.15.27:0/1296577740 learned_addr learned my addr 172.21.15.27:0/1296577740 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:57.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b2a00640 1 -- 172.21.15.27:0/1296577740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41039b0 msgr2=0x7f02b4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b2a00640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41039b0 0x7f02b4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b2a00640 1 -- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b4109c40 msgr2=0x7f02b4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b2a00640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b4109c40 0x7f02b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.382+0000 7f02b2a00640 1 -- 172.21.15.27:0/1296577740 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b41a85d0 con 0x7f02b41092f0 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.383+0000 7f02b2a00640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41092f0 0x7f02b4075c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f02b406b990 tx=0x7f02a000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.383+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000cc40 con 0x7f02b41092f0 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.383+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02a000cde0 con 0x7f02b41092f0 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.383+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b41a88c0 con 0x7f02b41092f0 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.383+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0018780 con 0x7f02b41092f0 2024-09-21T01:32:57.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.383+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b41a8e50 con 0x7f02b41092f0 2024-09-21T01:32:57.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.385+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b4103dc0 con 0x7f02b41092f0 2024-09-21T01:32:57.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.387+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f02a0010a90 con 0x7f02b41092f0 2024-09-21T01:32:57.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.388+0000 7f02abe00640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f02980b6700 0x7f02980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:57.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.389+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7f02a00e7ff0 con 0x7f02b41092f0 2024-09-21T01:32:57.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.389+0000 7f02b3400640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f02980b6700 0x7f02980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:57.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.389+0000 7f02b3400640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f02980b6700 0x7f02980b8c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009230 tx=0x7f02a40058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:57.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.390+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f02a00ae130 con 0x7f02b41092f0 2024-09-21T01:32:57.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.630+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f02b402e640 con 0x7f02b41092f0 2024-09-21T01:32:57.643 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.642+0000 7f02abe00640 1 -- 172.21.15.27:0/1296577740 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f02a00b2510 con 0x7f02b41092f0 2024-09-21T01:32:57.643 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T01:32:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.645+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f02980b6700 msgr2=0x7f02980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.645+0000 7f02b9000640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f02980b6700 0x7f02980b8c10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009230 tx=0x7f02a40058d0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.645+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41092f0 msgr2=0x7f02b4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.645+0000 7f02b9000640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41092f0 0x7f02b4075c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f02b406b990 tx=0x7f02a000ee50 comp rx=0 tx=0).stop 2024-09-21T01:32:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 shutdown_connections 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f02980b6700 0x7f02980b8c10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02b4109c40 0x7f02b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02b41092f0 0x7f02b4075c50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 --2- 172.21.15.27:0/1296577740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f02b41039b0 0x7f02b4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 >> 172.21.15.27:0/1296577740 conn(0x7f02b40ff440 msgr2=0x7f02b4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 shutdown_connections 2024-09-21T01:32:57.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:57.646+0000 7f02b9000640 1 -- 172.21.15.27:0/1296577740 wait complete. 2024-09-21T01:32:57.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d[72322]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:57.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d[72322]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:57.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mds[72326]: -- 172.21.15.27:0/3093043224 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8980af860 con 0x55a89c257180 2024-09-21T01:32:57.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d[72322]: starting mds.d at 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g[73090]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g[73090]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mds[73104]: -- 172.21.15.27:0/1248894897 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bb48cdc20 con 0x555bb8a75600 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g[73090]: starting mds.g at 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j[71942]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j[71942]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mds[71964]: -- 172.21.15.27:0/901346260 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b019b7860 con 0x555b05b5f180 2024-09-21T01:32:57.932 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j[71942]: starting mds.j at 2024-09-21T01:32:57.933 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a[30371]: 2024-09-21T01:32:57.632+0000 7f56f3e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: starting mds.b at 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1296577740' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T01:32:58.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mon[34486]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h[63132]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h[63132]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mds[63136]: -- 172.21.15.89:0/2686559431 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bfb51bbc20 con 0x55bfb9363180 2024-09-21T01:32:58.001 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h[63132]: starting mds.h at 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k[62578]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k[62578]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mds[62582]: -- 172.21.15.89:0/1107986212 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe72e77860 con 0x55fe7701f600 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k[62578]: starting mds.k at 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e[62857]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e[62857]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mds[62861]: -- 172.21.15.89:0/2830307060 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dffaa86b60 con 0x55dffec25600 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mds[62861]: --2- 172.21.15.89:0/2830307060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dffec25600 0x55dfff81c580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dffec36ba0 tx=0x55dffeb5c540 comp rx=0 tx=0).write_message sending message m=0x55dffeaea540 seq=3 auth(proto 2 2 bytes epoch 0) 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-mds[62861]: -- 172.21.15.89:0/2830307060 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dffaa7d860 con 0x55dffec25600 2024-09-21T01:32:58.002 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:32:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e[62857]: starting mds.e at 2024-09-21T01:32:58.148 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c[63802]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.148 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c[63802]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.148 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mds[63806]: -- 172.21.15.98:0/2873429324 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4c2e77860 con 0x55f4c701fa80 2024-09-21T01:32:58.148 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c[63802]: starting mds.c at 2024-09-21T01:32:58.148 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f[63538]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f[63538]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mds[63542]: -- 172.21.15.98:0/2746064480 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f6314b860 con 0x564f672f3180 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f[63538]: starting mds.f at 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1296577740' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T01:32:58.149 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T01:32:58.150 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mon[34372]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T01:32:58.150 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i[64075]: ignoring --setuser ceph since I am not root 2024-09-21T01:32:58.150 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i[64075]: ignoring --setgroup ceph since I am not root 2024-09-21T01:32:58.150 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-mds[64079]: -- 172.21.15.98:0/2417790818 <== mon.1 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564ead4d860 con 0x5564eeef5180 2024-09-21T01:32:58.150 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:32:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i[64075]: starting mds.i at 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: pgmap v724: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1296577740' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T01:32:58.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:57 smithi027 ceph-mon[30375]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T01:32:58.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T01:32:58.920 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:32:59.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 -- 172.21.15.27:0/378041662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc10de10 msgr2=0x7f70fc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 --2- 172.21.15.27:0/378041662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc10de10 0x7f70fc110300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009be0 tx=0x7f70f402a6c0 comp rx=0 tx=0).stop 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 -- 172.21.15.27:0/378041662 shutdown_connections 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 --2- 172.21.15.27:0/378041662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc10de10 0x7f70fc110300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 --2- 172.21.15.27:0/378041662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70fc105300 0x7f70fc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 --2- 172.21.15.27:0/378041662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 -- 172.21.15.27:0/378041662 >> 172.21.15.27:0/378041662 conn(0x7f70fc079760 msgr2=0x7f70fc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 -- 172.21.15.27:0/378041662 shutdown_connections 2024-09-21T01:32:59.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.620+0000 7f7102200640 1 -- 172.21.15.27:0/378041662 wait complete. 2024-09-21T01:32:59.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.621+0000 7f7102200640 1 Processor -- start 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 -- start start 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc105300 0x7f70fc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70fc10de10 0x7f70fc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc113a90 con 0x7f70fc1049b0 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc113910 con 0x7f70fc105300 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f7102200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc113c10 con 0x7f70fc10de10 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f70fb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f70fb400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58946/0 (socket says 172.21.15.27:58946) 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f70fb400640 1 -- 172.21.15.27:0/39101283 learned_addr learned my addr 172.21.15.27:0/39101283 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:32:59.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.622+0000 7f70f2a00640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc105300 0x7f70fc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70fbe00640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70fc10de10 0x7f70fc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70f2a00640 1 -- 172.21.15.27:0/39101283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70fc10de10 msgr2=0x7f70fc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70f2a00640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70fc10de10 0x7f70fc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70f2a00640 1 -- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 msgr2=0x7f70fc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70f2a00640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70f2a00640 1 -- 172.21.15.27:0/39101283 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc1a0430 con 0x7f70fc105300 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70fb400640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:32:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.623+0000 7f70f2a00640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc105300 0x7f70fc19aa60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f70fc06b950 tx=0x7f70e400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.624+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400cc40 con 0x7f70fc105300 2024-09-21T01:32:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.624+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc1a0720 con 0x7f70fc105300 2024-09-21T01:32:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.624+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70e40044d0 con 0x7f70fc105300 2024-09-21T01:32:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.624+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e401c070 con 0x7f70fc105300 2024-09-21T01:32:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.624+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc1a6e80 con 0x7f70fc105300 2024-09-21T01:32:59.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.625+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc06c2c0 con 0x7f70fc105300 2024-09-21T01:32:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.628+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f70e4005020 con 0x7f70fc105300 2024-09-21T01:32:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.630+0000 7f70f8c00640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f70e00b6680 0x7f70e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:32:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.630+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9832+0+0 (secure 0 0 0) 0x7f70e40e7c10 con 0x7f70fc105300 2024-09-21T01:32:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.630+0000 7f70fb400640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f70e00b6680 0x7f70e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:32:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.631+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70e40adc70 con 0x7f70fc105300 2024-09-21T01:32:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.631+0000 7f70fb400640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f70e00b6680 0x7f70e00b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0096c0 tx=0x7f70ec009260 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:32:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:32:59.871+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f70fc19b5f0 con 0x7f70fc105300 2024-09-21T01:32:59.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6832/843639205,v1:172.21.15.89:6834/843639205] up:boot 2024-09-21T01:32:59.872 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6834/3797324420,v1:172.21.15.98:6835/3797324420] up:boot 2024-09-21T01:32:59.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6833/1944474495,v1:172.21.15.89:6837/1944474495] up:boot 2024-09-21T01:32:59.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6836/3053050088,v1:172.21.15.89:6839/3053050088] up:boot 2024-09-21T01:32:59.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.89:6835/1880834,v1:172.21.15.89:6838/1880834] up:boot 2024-09-21T01:32:59.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6836/1261006723,v1:172.21.15.98:6838/1261006723] up:boot 2024-09-21T01:32:59.873 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.98:6837/3484685552,v1:172.21.15.98:6839/3484685552] up:boot 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6834/2105524679,v1:172.21.15.27:6836/2105524679] up:boot 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6835/1852222607,v1:172.21.15.27:6838/1852222607] up:boot 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: mds.? [v2:172.21.15.27:6837/826290298,v1:172.21.15.27:6839/826290298] up:boot 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:32:59.874 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:32:59.875 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:32:59.875 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:32:59.875 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:32:59.875 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:32:59 smithi027 ceph-mon[30375]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T01:32:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6832/843639205,v1:172.21.15.89:6834/843639205] up:boot 2024-09-21T01:32:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6834/3797324420,v1:172.21.15.98:6835/3797324420] up:boot 2024-09-21T01:32:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6833/1944474495,v1:172.21.15.89:6837/1944474495] up:boot 2024-09-21T01:32:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6836/3053050088,v1:172.21.15.89:6839/3053050088] up:boot 2024-09-21T01:32:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.89:6835/1880834,v1:172.21.15.89:6838/1880834] up:boot 2024-09-21T01:32:59.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6836/1261006723,v1:172.21.15.98:6838/1261006723] up:boot 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.98:6837/3484685552,v1:172.21.15.98:6839/3484685552] up:boot 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6834/2105524679,v1:172.21.15.27:6836/2105524679] up:boot 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6835/1852222607,v1:172.21.15.27:6838/1852222607] up:boot 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: mds.? [v2:172.21.15.27:6837/826290298,v1:172.21.15.27:6839/826290298] up:boot 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:32:59.899 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:32:59.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:32:59.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:32:59.900 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:32:59 smithi098 ceph-mon[34372]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6832/843639205,v1:172.21.15.89:6834/843639205] up:boot 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6834/3797324420,v1:172.21.15.98:6835/3797324420] up:boot 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6833/1944474495,v1:172.21.15.89:6837/1944474495] up:boot 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6836/3053050088,v1:172.21.15.89:6839/3053050088] up:boot 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.89:6835/1880834,v1:172.21.15.89:6838/1880834] up:boot 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6836/1261006723,v1:172.21.15.98:6838/1261006723] up:boot 2024-09-21T01:33:00.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.98:6837/3484685552,v1:172.21.15.98:6839/3484685552] up:boot 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6834/2105524679,v1:172.21.15.27:6836/2105524679] up:boot 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6835/1852222607,v1:172.21.15.27:6838/1852222607] up:boot 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: mds.? [v2:172.21.15.27:6837/826290298,v1:172.21.15.27:6839/826290298] up:boot 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T01:33:00.001 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T01:33:00.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T01:33:00.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: from='mgr.24110 172.21.15.27:0/299347728' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T01:33:00.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:32:59 smithi089 ceph-mon[34486]: pgmap v726: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T01:33:00.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.657+0000 7f70f8c00640 1 -- 172.21.15.27:0/39101283 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f70e40b2050 con 0x7f70fc105300 2024-09-21T01:33:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f70e00b6680 msgr2=0x7f70e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f70e00b6680 0x7f70e00b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0096c0 tx=0x7f70ec009260 comp rx=0 tx=0).stop 2024-09-21T01:33:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc105300 msgr2=0x7f70fc19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc105300 0x7f70fc19aa60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f70fc06b950 tx=0x7f70e400ee50 comp rx=0 tx=0).stop 2024-09-21T01:33:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 shutdown_connections 2024-09-21T01:33:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f70e00b6680 0x7f70e00b8b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70fc10de10 0x7f70fc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f70fc105300 0x7f70fc19aa60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 --2- 172.21.15.27:0/39101283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70fc1049b0 0x7f70fc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 >> 172.21.15.27:0/39101283 conn(0x7f70fc079760 msgr2=0x7f70fc107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 shutdown_connections 2024-09-21T01:33:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:00.660+0000 7f7102200640 1 -- 172.21.15.27:0/39101283 wait complete. 2024-09-21T01:33:00.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:00 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/39101283' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:33:00.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:00 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:33:00.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:00 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/39101283' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:33:00.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:00 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:33:00.999 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:00 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/39101283' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:33:01.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:00 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T01:33:01.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T01:33:01.822 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:33:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:01 smithi027 ceph-mon[30375]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T01:33:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:01 smithi027 ceph-mon[30375]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T01:33:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:01 smithi027 ceph-mon[30375]: Cluster is now healthy 2024-09-21T01:33:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:01 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:33:01.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:01 smithi027 ceph-mon[30375]: fsmap 12 up:standby 2024-09-21T01:33:01.932 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:01 smithi027 ceph-mon[30375]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 307 B/s wr, 8 op/s 2024-09-21T01:33:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:01 smithi089 ceph-mon[34486]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T01:33:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:01 smithi089 ceph-mon[34486]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T01:33:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:01 smithi089 ceph-mon[34486]: Cluster is now healthy 2024-09-21T01:33:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:01 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:33:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:01 smithi089 ceph-mon[34486]: fsmap 12 up:standby 2024-09-21T01:33:02.000 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:01 smithi089 ceph-mon[34486]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 307 B/s wr, 8 op/s 2024-09-21T01:33:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:01 smithi098 ceph-mon[34372]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T01:33:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:01 smithi098 ceph-mon[34372]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T01:33:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:01 smithi098 ceph-mon[34372]: Cluster is now healthy 2024-09-21T01:33:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:01 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T01:33:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:01 smithi098 ceph-mon[34372]: fsmap 12 up:standby 2024-09-21T01:33:02.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:01 smithi098 ceph-mon[34372]: pgmap v727: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 307 B/s wr, 8 op/s 2024-09-21T01:33:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 -- 172.21.15.27:0/1522128528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 msgr2=0x7f6c7410d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 --2- 172.21.15.27:0/1522128528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 0x7f6c7410d670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009900 tx=0x7f6c5c02a3f0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 -- 172.21.15.27:0/1522128528 shutdown_connections 2024-09-21T01:33:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 --2- 172.21.15.27:0/1522128528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c7410dc30 0x7f6c74110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 --2- 172.21.15.27:0/1522128528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 0x7f6c7410d670 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 --2- 172.21.15.27:0/1522128528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c74104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 -- 172.21.15.27:0/1522128528 >> 172.21.15.27:0/1522128528 conn(0x7f6c740fe2d0 msgr2=0x7f6c74100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 -- 172.21.15.27:0/1522128528 shutdown_connections 2024-09-21T01:33:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.515+0000 7f6c73e00640 1 -- 172.21.15.27:0/1522128528 wait complete. 2024-09-21T01:33:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.516+0000 7f6c73e00640 1 Processor -- start 2024-09-21T01:33:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 -- start start 2024-09-21T01:33:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c7419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 0x7f6c7419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c7410dc30 0x7f6c7419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74113890 con 0x7f6c74105120 2024-09-21T01:33:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74113710 con 0x7f6c741047d0 2024-09-21T01:33:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73e00640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74113a10 con 0x7f6c7410dc30 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c72a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c7419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c6be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 0x7f6c7419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c72a00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c7419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:55046/0 (socket says 172.21.15.27:55046) 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c72a00640 1 -- 172.21.15.27:0/618603556 learned_addr learned my addr 172.21.15.27:0/618603556 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.517+0000 7f6c73400640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c7410dc30 0x7f6c7419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.518+0000 7f6c72a00640 1 -- 172.21.15.27:0/618603556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c7410dc30 msgr2=0x7f6c7419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.518+0000 7f6c72a00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c7410dc30 0x7f6c7419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.518+0000 7f6c72a00640 1 -- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 msgr2=0x7f6c7419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.518+0000 7f6c72a00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 0x7f6c7419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.518+0000 7f6c72a00640 1 -- 172.21.15.27:0/618603556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a0230 con 0x7f6c741047d0 2024-09-21T01:33:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.518+0000 7f6c72a00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c7419a2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e9f0 tx=0x7f6c6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:33:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.519+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400cd80 con 0x7f6c741047d0 2024-09-21T01:33:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.519+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741a0520 con 0x7f6c741047d0 2024-09-21T01:33:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.519+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c640108e0 con 0x7f6c741047d0 2024-09-21T01:33:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.519+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64018980 con 0x7f6c741047d0 2024-09-21T01:33:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.519+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a6c80 con 0x7f6c741047d0 2024-09-21T01:33:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.520+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74104be0 con 0x7f6c741047d0 2024-09-21T01:33:02.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.523+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6c64010430 con 0x7f6c741047d0 2024-09-21T01:33:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.524+0000 7f6c6b400640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c580b6680 0x7f6c580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.524+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9832+0+0 (secure 0 0 0) 0x7f6c640e8450 con 0x7f6c741047d0 2024-09-21T01:33:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.524+0000 7f6c6be00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c580b6680 0x7f6c580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:02.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.525+0000 7f6c6be00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c580b6680 0x7f6c580b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009230 tx=0x7f6c5c031400 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:33:02.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.526+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6c640b2800 con 0x7f6c741047d0 2024-09-21T01:33:02.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.725+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c741a6f20 con 0x7f6c741047d0 2024-09-21T01:33:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.786+0000 7f6c6b400640 1 -- 172.21.15.27:0/618603556 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f6c640b7740 con 0x7f6c741047d0 2024-09-21T01:33:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-09-21T01:33:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c580b6680 msgr2=0x7f6c580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c580b6680 0x7f6c580b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009230 tx=0x7f6c5c031400 comp rx=0 tx=0).stop 2024-09-21T01:33:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 msgr2=0x7f6c7419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c7419a2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e9f0 tx=0x7f6c6400eeb0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 shutdown_connections 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f6c580b6680 0x7f6c580b8b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c7410dc30 0x7f6c7419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c74105120 0x7f6c7419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 --2- 172.21.15.27:0/618603556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6c741047d0 0x7f6c7419a2c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.788+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 >> 172.21.15.27:0/618603556 conn(0x7f6c740fe2d0 msgr2=0x7f6c74106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.789+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 shutdown_connections 2024-09-21T01:33:02.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:02.789+0000 7f6c73e00640 1 -- 172.21.15.27:0/618603556 wait complete. 2024-09-21T01:33:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:02 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/618603556' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:03.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:02 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:02 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/618603556' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:03.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:02 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:03.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:02 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/618603556' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:03.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:02 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:03.581 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T01:33:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:03 smithi027 ceph-mon[30375]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T01:33:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:03 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:03.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:03 smithi027 ceph-mon[30375]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T01:33:03.972 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:33:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:03 smithi098 ceph-mon[34372]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T01:33:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:03 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:04.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:03 smithi098 ceph-mon[34372]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T01:33:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:03 smithi089 ceph-mon[34486]: pgmap v728: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T01:33:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:03 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:04.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:03 smithi089 ceph-mon[34486]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 -- 172.21.15.27:0/1463777065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 msgr2=0x7f4be4112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 --2- 172.21.15.27:0/1463777065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 0x7f4be4112850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009ca0 tx=0x7f4be002a8c0 comp rx=0 tx=0).stop 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 -- 172.21.15.27:0/1463777065 shutdown_connections 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 --2- 172.21.15.27:0/1463777065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 0x7f4be4112850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 --2- 172.21.15.27:0/1463777065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4be410b6d0 0x7f4be410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 --2- 172.21.15.27:0/1463777065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 -- 172.21.15.27:0/1463777065 >> 172.21.15.27:0/1463777065 conn(0x7f4be4101860 msgr2=0x7f4be4103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 -- 172.21.15.27:0/1463777065 shutdown_connections 2024-09-21T01:33:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.747+0000 7f4bec800640 1 -- 172.21.15.27:0/1463777065 wait complete. 2024-09-21T01:33:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.748+0000 7f4bec800640 1 Processor -- start 2024-09-21T01:33:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 -- start start 2024-09-21T01:33:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4be410b6d0 0x7f4be419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 0x7f4be41a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4115ce0 con 0x7f4be410c020 2024-09-21T01:33:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4115b60 con 0x7f4be4105d90 2024-09-21T01:33:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bec800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4115e60 con 0x7f4be410b6d0 2024-09-21T01:33:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4be9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4be9e00640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:55066/0 (socket says 172.21.15.27:55066) 2024-09-21T01:33:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4bea800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 0x7f4be41a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4be9e00640 1 -- 172.21.15.27:0/1142005704 learned_addr learned my addr 172.21.15.27:0/1142005704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:33:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.749+0000 7f4be9400640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4be410b6d0 0x7f4be419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4be9e00640 1 -- 172.21.15.27:0/1142005704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4be410b6d0 msgr2=0x7f4be419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4be9e00640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4be410b6d0 0x7f4be419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4be9e00640 1 -- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 msgr2=0x7f4be41a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4be9e00640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 0x7f4be41a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4be9e00640 1 -- 172.21.15.27:0/1142005704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be41a25e0 con 0x7f4be4105d90 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4be9e00640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be419c670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800e9f0 tx=0x7f4bd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.750+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800cd80 con 0x7f4be4105d90 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.751+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be41a28d0 con 0x7f4be4105d90 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.751+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd80108e0 con 0x7f4be4105d90 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.751+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8018980 con 0x7f4be4105d90 2024-09-21T01:33:04.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.751+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be41a9030 con 0x7f4be4105d90 2024-09-21T01:33:04.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.752+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be41061a0 con 0x7f4be4105d90 2024-09-21T01:33:04.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.755+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f4bd8010430 con 0x7f4be4105d90 2024-09-21T01:33:04.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.756+0000 7f4bdea00640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4bcc0b6700 0x7f4bcc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:04.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.756+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9278+0+0 (secure 0 0 0) 0x7f4bd80e81a0 con 0x7f4be4105d90 2024-09-21T01:33:04.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.757+0000 7f4be9400640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4bcc0b6700 0x7f4bcc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:04.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.757+0000 7f4be9400640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4bcc0b6700 0x7f4bcc0b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4be41050b0 tx=0x7f4bd4005f90 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:33:04.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.758+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4bd80b2780 con 0x7f4be4105d90 2024-09-21T01:33:04.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:04.961+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4be402e640 con 0x7f4be4105d90 2024-09-21T01:33:05.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.806+0000 7f4bdea00640 1 -- 172.21.15.27:0/1142005704 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f4bd80b76c0 con 0x7f4be4105d90 2024-09-21T01:33:05.808 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4bcc0b6700 msgr2=0x7f4bcc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4bcc0b6700 0x7f4bcc0b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4be41050b0 tx=0x7f4bd4005f90 comp rx=0 tx=0).stop 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 msgr2=0x7f4be419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be419c670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800e9f0 tx=0x7f4bd800eeb0 comp rx=0 tx=0).stop 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 shutdown_connections 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f4bcc0b6700 0x7f4bcc0b8c10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:05.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be410c020 0x7f4be41a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4be410b6d0 0x7f4be419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 --2- 172.21.15.27:0/1142005704 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4be4105d90 0x7f4be419c670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.811+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 >> 172.21.15.27:0/1142005704 conn(0x7f4be4101860 msgr2=0x7f4be4108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.812+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 shutdown_connections 2024-09-21T01:33:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:05.812+0000 7f4bec800640 1 -- 172.21.15.27:0/1142005704 wait complete. 2024-09-21T01:33:06.052 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:05 smithi089 ceph-mon[34486]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:06.052 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:05 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1142005704' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:06.052 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:05 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:05 smithi098 ceph-mon[34372]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:05 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1142005704' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:06.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:05 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:05 smithi027 ceph-mon[30375]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:05 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1142005704' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:06.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:05 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:06.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T01:33:06.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:06 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:06.931 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:06 smithi027 ceph-mon[30375]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T01:33:07.011 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T01:33:07.064 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:06 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:07.064 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:06 smithi089 ceph-mon[34486]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T01:33:07.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:06 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:07.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:06 smithi098 ceph-mon[34372]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 -- 172.21.15.27:0/718675567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 msgr2=0x7f37e0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 --2- 172.21.15.27:0/718675567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e0103f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009900 tx=0x7f37d002a3f0 comp rx=0 tx=0).stop 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 -- 172.21.15.27:0/718675567 shutdown_connections 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 --2- 172.21.15.27:0/718675567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 0x7f37e0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 --2- 172.21.15.27:0/718675567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 0x7f37e01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 --2- 172.21.15.27:0/718675567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e0103f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 -- 172.21.15.27:0/718675567 >> 172.21.15.27:0/718675567 conn(0x7f37e007b850 msgr2=0x7f37e007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:07.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 -- 172.21.15.27:0/718675567 shutdown_connections 2024-09-21T01:33:07.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.654+0000 7f37e6200640 1 -- 172.21.15.27:0/718675567 wait complete. 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.655+0000 7f37e6200640 1 Processor -- start 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.655+0000 7f37e6200640 1 -- start start 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37e6200640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37e6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 0x7f37e019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37e6200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 0x7f37e019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37e6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0113ab0 con 0x7f37e01094a0 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37e6200640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0113930 con 0x7f37e0103b60 2024-09-21T01:33:07.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37e6200640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0113c30 con 0x7f37e0109df0 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37df400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37dea00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 0x7f37e019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37df400640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.27:35444/0 (socket says 172.21.15.27:35444) 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37df400640 1 -- 172.21.15.27:0/1767596498 learned_addr learned my addr 172.21.15.27:0/1767596498 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37dfe00640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 0x7f37e019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37dfe00640 1 -- 172.21.15.27:0/1767596498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 msgr2=0x7f37e019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37dfe00640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.656+0000 7f37dfe00640 1 -- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 msgr2=0x7f37e019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37dfe00640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 0x7f37e019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37dfe00640 1 -- 172.21.15.27:0/1767596498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01a0430 con 0x7f37e0109df0 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37df400640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:33:07.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37dea00640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 0x7f37e019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T01:33:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37dfe00640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 0x7f37e019fd00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009fd0 tx=0x7f37d800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:33:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800ce20 con 0x7f37e0109df0 2024-09-21T01:33:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d8005020 con 0x7f37e0109df0 2024-09-21T01:33:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01a0720 con 0x7f37e0109df0 2024-09-21T01:33:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.657+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8010690 con 0x7f37e0109df0 2024-09-21T01:33:07.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.658+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01a6e80 con 0x7f37e0109df0 2024-09-21T01:33:07.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.659+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e0103f70 con 0x7f37e0109df0 2024-09-21T01:33:07.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.661+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f37d80054d0 con 0x7f37e0109df0 2024-09-21T01:33:07.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.663+0000 7f37d7e00640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37c40b6680 0x7f37c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T01:33:07.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.663+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8711+0+0 (secure 0 0 0) 0x7f37d80e7dd0 con 0x7f37e0109df0 2024-09-21T01:33:07.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.663+0000 7f37df400640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37c40b6680 0x7f37c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T01:33:07.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.664+0000 7f37df400640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37c40b6680 0x7f37c40b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f37d0031440 tx=0x7f37d00058d0 comp rx=0 tx=0).ready entity=mgr.24110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T01:33:07.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.665+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37d80ae260 con 0x7f37e0109df0 2024-09-21T01:33:07.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:07.865+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37e019b5f0 con 0x7f37e0109df0 2024-09-21T01:33:08.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:07 smithi098 ceph-mon[34372]: pgmap v732: 33 pgs: 33 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:08.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:07 smithi027 ceph-mon[30375]: pgmap v732: 33 pgs: 33 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:08.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:07 smithi089 ceph-mon[34486]: pgmap v732: 33 pgs: 33 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.816+0000 7f37d7e00640 1 -- 172.21.15.27:0/1767596498 <== mon.1 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v102) ==== 178+0+0 (secure 0 0 0) 0x7f37e019b5f0 con 0x7f37e0109df0 2024-09-21T01:33:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data_ec' removed 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.821+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37c40b6680 msgr2=0x7f37c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.821+0000 7f37e6200640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37c40b6680 0x7f37c40b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f37d0031440 tx=0x7f37d00058d0 comp rx=0 tx=0).stop 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.821+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 msgr2=0x7f37e019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.821+0000 7f37e6200640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 0x7f37e019fd00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009fd0 tx=0x7f37d800ef10 comp rx=0 tx=0).stop 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 shutdown_connections 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:6800/39380413,v1:172.21.15.27:6801/39380413] conn(0x7f37c40b6680 0x7f37c40b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e0109df0 0x7f37e019fd00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37e01094a0 0x7f37e019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 --2- 172.21.15.27:0/1767596498 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e0103b60 0x7f37e019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T01:33:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 >> 172.21.15.27:0/1767596498 conn(0x7f37e007b850 msgr2=0x7f37e0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T01:33:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 shutdown_connections 2024-09-21T01:33:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-09-21T01:33:08.822+0000 7f37e6200640 1 -- 172.21.15.27:0/1767596498 wait complete. 2024-09-21T01:33:09.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:08 smithi098 ceph-mon[34372]: from='client.? 172.21.15.27:0/1767596498' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:09.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:08 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:09.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:08 smithi027 ceph-mon[30375]: from='client.? 172.21.15.27:0/1767596498' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:09.181 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:08 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:09.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:08 smithi089 ceph-mon[34486]: from='client.? 172.21.15.27:0/1767596498' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:09.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:08 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T01:33:09.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:33:09.755 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:33:09.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:33:09.830 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T01:33:09.830 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T01:33:09.881 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T01:33:09.902 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T01:33:09.928 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T01:33:09.929 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T01:33:09.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a 2024-09-21T01:33:09.936 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:09 smithi027 ceph-mon[30375]: pgmap v733: 33 pgs: 33 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:09.936 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:09 smithi027 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:09.936 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:09 smithi027 ceph-mon[30375]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T01:33:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:09 smithi098 ceph-mon[34372]: pgmap v733: 33 pgs: 33 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:09 smithi098 ceph-mon[34372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:10.148 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:09 smithi098 ceph-mon[34372]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T01:33:10.249 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:09 smithi089 ceph-mon[34486]: pgmap v733: 33 pgs: 33 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T01:33:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:09 smithi089 ceph-mon[34486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T01:33:10.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:09 smithi089 ceph-mon[34486]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T01:33:10.408 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:10 smithi027 systemd[1]: Stopping Ceph mon.a for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:10.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a[30371]: 2024-09-21T01:33:10.406+0000 7f56fac00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:10.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a[30371]: 2024-09-21T01:33:10.406+0000 7f56fac00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T01:33:10.681 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:10 smithi027 podman[94745]: 2024-09-21 01:33:10.551996943 +0000 UTC m=+0.267123079 container died 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:33:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:11 smithi027 podman[94745]: 2024-09-21 01:33:11.271810944 +0000 UTC m=+0.986937079 container cleanup 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T01:33:11.431 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:11 smithi027 bash[94745]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a 2024-09-21T01:33:11.864 INFO:journalctl@ceph.mon.a.smithi027.stdout:Sep 21 01:33:11 smithi027 podman[94757]: 2024-09-21 01:33:11.573646067 +0000 UTC m=+1.013311918 container remove 14af1170be195fcadf6e22fda608ff922154edeb3ed3406fc22d4baf546f7d00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T01:33:11.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T01:33:12.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:12.078 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T01:33:12.078 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T01:33:12.078 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.b 2024-09-21T01:33:12.399 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:12 smithi089 systemd[1]: Stopping Ceph mon.b for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:12.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-b[34482]: 2024-09-21T01:33:12.398+0000 7f21eb521640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:12.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-b[34482]: 2024-09-21T01:33:12.398+0000 7f21eb521640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T01:33:12.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:12 smithi089 podman[67012]: 2024-09-21 01:33:12.501090307 +0000 UTC m=+0.229906366 container died 2258882a6026f5942c2183da8e71f7663469a3e8b028b6724c88b3a322be6ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T01:33:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:12 smithi089 podman[67012]: 2024-09-21 01:33:12.965625823 +0000 UTC m=+0.694441869 container cleanup 2258882a6026f5942c2183da8e71f7663469a3e8b028b6724c88b3a322be6ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T01:33:13.250 INFO:journalctl@ceph.mon.b.smithi089.stdout:Sep 21 01:33:12 smithi089 bash[67012]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-b 2024-09-21T01:33:13.850 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T01:33:13.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:13.893 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T01:33:13.894 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T01:33:13.894 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.c 2024-09-21T01:33:14.221 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:13 smithi098 systemd[1]: Stopping Ceph mon.c for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:14.573 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c[34368]: 2024-09-21T01:33:14.221+0000 7f27e5909640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:14.573 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c[34368]: 2024-09-21T01:33:14.221+0000 7f27e5909640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T01:33:14.573 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:14 smithi098 podman[67445]: 2024-09-21 01:33:14.242269238 +0000 UTC m=+0.141607675 container died ff8ee050bf7aa4aa457158054313b29fc4fa60fa89d18a70d62b909c732cc555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T01:33:14.897 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:14 smithi098 podman[67445]: 2024-09-21 01:33:14.672360732 +0000 UTC m=+0.571699165 container cleanup ff8ee050bf7aa4aa457158054313b29fc4fa60fa89d18a70d62b909c732cc555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:33:14.898 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:14 smithi098 bash[67445]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c 2024-09-21T01:33:15.149 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T01:33:15.171 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:14 smithi098 podman[67459]: 2024-09-21 01:33:14.899366637 +0000 UTC m=+0.648765725 container remove ff8ee050bf7aa4aa457158054313b29fc4fa60fa89d18a70d62b909c732cc555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mon-c, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T01:33:15.171 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:15 smithi098 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.c.service: Deactivated successfully. 2024-09-21T01:33:15.171 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:15 smithi098 systemd[1]: Stopped Ceph mon.c for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:33:15.171 INFO:journalctl@ceph.mon.c.smithi098.stdout:Sep 21 01:33:15 smithi098 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mon.c.service: Consumed 15.169s CPU time. 2024-09-21T01:33:15.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:15.193 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T01:33:15.193 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T01:33:15.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x 2024-09-21T01:33:15.681 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:15 smithi027 systemd[1]: Stopping Ceph mgr.x for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:16.181 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:15 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:33:15.679+0000 7f65fa600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:16.181 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:15 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x[30601]: 2024-09-21T01:33:15.679+0000 7f65fa600640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T01:33:16.181 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:15 smithi027 podman[94868]: 2024-09-21 01:33:15.773327964 +0000 UTC m=+0.289982780 container died a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:33:16.622 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:16 smithi027 podman[94868]: 2024-09-21 01:33:16.394578853 +0000 UTC m=+0.911233668 container cleanup a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T01:33:16.622 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:16 smithi027 bash[94868]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x 2024-09-21T01:33:16.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Sep 21 01:33:16 smithi027 podman[94880]: 2024-09-21 01:33:16.621895086 +0000 UTC m=+0.836511512 container remove a032899fd50c03f7661dfd344e9cec51b7d4ab75152670ce8fc6ce52aad43230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T01:33:16.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T01:33:16.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:16.975 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T01:33:16.975 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T01:33:16.976 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.y 2024-09-21T01:33:17.308 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:17 smithi089 systemd[1]: Stopping Ceph mgr.y for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:17.622 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:17 smithi089 podman[67136]: 2024-09-21 01:33:17.308479754 +0000 UTC m=+0.135375861 container died 789eb5d1a42804227e5c16cfd8c98b74ea6e4d78ad688f515da9bb973069d7a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T01:33:17.973 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:17 smithi089 podman[67136]: 2024-09-21 01:33:17.729872394 +0000 UTC m=+0.556768503 container cleanup 789eb5d1a42804227e5c16cfd8c98b74ea6e4d78ad688f515da9bb973069d7a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T01:33:17.974 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:17 smithi089 bash[67136]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-y 2024-09-21T01:33:18.202 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T01:33:18.225 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:17 smithi089 podman[67148]: 2024-09-21 01:33:17.97369312 +0000 UTC m=+0.656895785 container remove 789eb5d1a42804227e5c16cfd8c98b74ea6e4d78ad688f515da9bb973069d7a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-y, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T01:33:18.225 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:17 smithi089 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-21T01:33:18.225 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:18 smithi089 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.y.service: Failed with result 'exit-code'. 2024-09-21T01:33:18.225 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:18 smithi089 systemd[1]: Stopped Ceph mgr.y for fe42ec90-77b5-11ef-baf5-efdc52797490. 2024-09-21T01:33:18.225 INFO:journalctl@ceph.mgr.y.smithi089.stdout:Sep 21 01:33:18 smithi089 systemd[1]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.y.service: Consumed 7.240s CPU time. 2024-09-21T01:33:18.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:18.237 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T01:33:18.237 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T01:33:18.237 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.z 2024-09-21T01:33:18.581 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:33:18 smithi098 systemd[1]: Stopping Ceph mgr.z for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:18.878 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:33:18 smithi098 podman[67566]: 2024-09-21 01:33:18.581725213 +0000 UTC m=+0.150136269 container died 3a4296d6352babfb23c3f0e27a3fcd146a5f4c2be2019a6b6db699fb2bcc3c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:33:19.148 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:33:18 smithi098 podman[67566]: 2024-09-21 01:33:18.978119609 +0000 UTC m=+0.546530673 container cleanup 3a4296d6352babfb23c3f0e27a3fcd146a5f4c2be2019a6b6db699fb2bcc3c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:33:19.148 INFO:journalctl@ceph.mgr.z.smithi098.stdout:Sep 21 01:33:18 smithi098 bash[67566]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mgr-z 2024-09-21T01:33:19.649 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T01:33:19.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:19.684 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T01:33:19.684 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T01:33:19.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.0 2024-09-21T01:33:20.181 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:19 smithi027 systemd[1]: Stopping Ceph osd.0 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:20.681 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0[47260]: 2024-09-21T01:33:20.352+0000 7f394be00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:20.681 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0[47260]: 2024-09-21T01:33:20.352+0000 7f394be00640 -1 osd.0 102 *** Got signal Terminated *** 2024-09-21T01:33:20.681 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0[47260]: 2024-09-21T01:33:20.352+0000 7f394be00640 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:33:26.114 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:25 smithi027 podman[94985]: 2024-09-21 01:33:25.7782052 +0000 UTC m=+5.790661362 container died 4438aacc1a14cf71af3e221e3d808033d14ff62f1d08e7e25e00618f1a7f20ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T01:33:26.431 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:26 smithi027 podman[94985]: 2024-09-21 01:33:26.293010157 +0000 UTC m=+6.305466319 container cleanup 4438aacc1a14cf71af3e221e3d808033d14ff62f1d08e7e25e00618f1a7f20ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T01:33:26.431 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:26 smithi027 bash[94985]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0 2024-09-21T01:33:26.931 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:26 smithi027 podman[95002]: 2024-09-21 01:33:26.527565028 +0000 UTC m=+0.736447717 container remove 4438aacc1a14cf71af3e221e3d808033d14ff62f1d08e7e25e00618f1a7f20ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:33:27.406 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:27 smithi027 podman[95092]: 2024-09-21 01:33:26.979087745 +0000 UTC m=+0.028129880 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:33:27.406 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:27 smithi027 podman[95092]: 2024-09-21 01:33:27.136910993 +0000 UTC m=+0.185953126 container create 4ac6c37fbfdd6c3f598bc55eb50cc258488f03a934af584b116d8109562b083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2) 2024-09-21T01:33:27.406 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:27 smithi027 podman[95092]: 2024-09-21 01:33:27.406524993 +0000 UTC m=+0.455567137 container init 4ac6c37fbfdd6c3f598bc55eb50cc258488f03a934af584b116d8109562b083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:33:27.679 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:27 smithi027 podman[95092]: 2024-09-21 01:33:27.413634213 +0000 UTC m=+0.462676347 container start 4ac6c37fbfdd6c3f598bc55eb50cc258488f03a934af584b116d8109562b083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T01:33:27.679 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:27 smithi027 podman[95092]: 2024-09-21 01:33:27.514649211 +0000 UTC m=+0.563691362 container attach 4ac6c37fbfdd6c3f598bc55eb50cc258488f03a934af584b116d8109562b083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T01:33:27.931 INFO:journalctl@ceph.osd.0.smithi027.stdout:Sep 21 01:33:27 smithi027 podman[95092]: 2024-09-21 01:33:27.680194939 +0000 UTC m=+0.729237087 container died 4ac6c37fbfdd6c3f598bc55eb50cc258488f03a934af584b116d8109562b083d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:33:28.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T01:33:28.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:28.546 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T01:33:28.547 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T01:33:28.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.1 2024-09-21T01:33:28.852 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:28 smithi089 systemd[1]: Stopping Ceph osd.1 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:29.250 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1[39082]: 2024-09-21T01:33:28.850+0000 7fc0ed59b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:29.250 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1[39082]: 2024-09-21T01:33:28.850+0000 7fc0ed59b640 -1 osd.1 102 *** Got signal Terminated *** 2024-09-21T01:33:29.250 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1[39082]: 2024-09-21T01:33:28.850+0000 7fc0ed59b640 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:33:34.601 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:34 smithi089 podman[67262]: 2024-09-21 01:33:34.290514826 +0000 UTC m=+5.555080534 container died 0612ce39ba7383130786899f9e1c9384007d6a20f87c60b8e6f200abc92893e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:33:34.953 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:34 smithi089 podman[67262]: 2024-09-21 01:33:34.701399833 +0000 UTC m=+5.965965533 container cleanup 0612ce39ba7383130786899f9e1c9384007d6a20f87c60b8e6f200abc92893e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T01:33:34.953 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:34 smithi089 bash[67262]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1 2024-09-21T01:33:35.249 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:34 smithi089 podman[67277]: 2024-09-21 01:33:34.953428524 +0000 UTC m=+0.660560074 container remove 0612ce39ba7383130786899f9e1c9384007d6a20f87c60b8e6f200abc92893e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:33:35.665 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:35 smithi089 podman[67366]: 2024-09-21 01:33:35.2674319 +0000 UTC m=+0.015941538 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:33:35.665 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:35 smithi089 podman[67366]: 2024-09-21 01:33:35.404268935 +0000 UTC m=+0.152778608 container create 4d2a5ede604ffb12ed22a6c68f203fc0db8bb747755c5138f11502dd1227c5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T01:33:36.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:35 smithi089 podman[67366]: 2024-09-21 01:33:35.665239401 +0000 UTC m=+0.413749034 container init 4d2a5ede604ffb12ed22a6c68f203fc0db8bb747755c5138f11502dd1227c5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:33:36.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:35 smithi089 podman[67366]: 2024-09-21 01:33:35.668680506 +0000 UTC m=+0.417190138 container start 4d2a5ede604ffb12ed22a6c68f203fc0db8bb747755c5138f11502dd1227c5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T01:33:36.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:35 smithi089 podman[67366]: 2024-09-21 01:33:35.757065439 +0000 UTC m=+0.505575075 container attach 4d2a5ede604ffb12ed22a6c68f203fc0db8bb747755c5138f11502dd1227c5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T01:33:36.000 INFO:journalctl@ceph.osd.1.smithi089.stdout:Sep 21 01:33:35 smithi089 podman[67366]: 2024-09-21 01:33:35.852128599 +0000 UTC m=+0.600638236 container died 4d2a5ede604ffb12ed22a6c68f203fc0db8bb747755c5138f11502dd1227c5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:33:36.547 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T01:33:36.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:36.587 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T01:33:36.587 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T01:33:36.587 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.2 2024-09-21T01:33:36.897 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:36 smithi098 systemd[1]: Stopping Ceph osd.2 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:37.398 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2[39550]: 2024-09-21T01:33:36.907+0000 7f218b7c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:37.398 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2[39550]: 2024-09-21T01:33:36.907+0000 7f218b7c8640 -1 osd.2 102 *** Got signal Terminated *** 2024-09-21T01:33:37.398 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2[39550]: 2024-09-21T01:33:36.907+0000 7f218b7c8640 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:33:42.577 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:42 smithi098 podman[67694]: 2024-09-21 01:33:42.260601062 +0000 UTC m=+5.478538197 container died 038ce13d32fecce9aa4335cdc0e673968ab83f1b3f75b0d7f980eae5048ffe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T01:33:42.898 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:42 smithi098 podman[67694]: 2024-09-21 01:33:42.683427323 +0000 UTC m=+5.901364405 container cleanup 038ce13d32fecce9aa4335cdc0e673968ab83f1b3f75b0d7f980eae5048ffe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T01:33:42.898 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:42 smithi098 bash[67694]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2 2024-09-21T01:33:43.340 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:42 smithi098 podman[67712]: 2024-09-21 01:33:42.9268182 +0000 UTC m=+0.659363440 container remove 038ce13d32fecce9aa4335cdc0e673968ab83f1b3f75b0d7f980eae5048ffe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T01:33:43.622 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:43 smithi098 podman[67800]: 2024-09-21 01:33:43.242683297 +0000 UTC m=+0.015762483 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:33:43.622 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:43 smithi098 podman[67800]: 2024-09-21 01:33:43.377731102 +0000 UTC m=+0.150810288 container create 8607f3d47ac52112be1aa22ea65b1e57291dd0413acaeaaf3854b52c325dc2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T01:33:43.622 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:43 smithi098 podman[67800]: 2024-09-21 01:33:43.622292339 +0000 UTC m=+0.395371525 container init 8607f3d47ac52112be1aa22ea65b1e57291dd0413acaeaaf3854b52c325dc2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T01:33:43.898 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:43 smithi098 podman[67800]: 2024-09-21 01:33:43.625931228 +0000 UTC m=+0.399010414 container start 8607f3d47ac52112be1aa22ea65b1e57291dd0413acaeaaf3854b52c325dc2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T01:33:43.898 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:43 smithi098 podman[67800]: 2024-09-21 01:33:43.730887608 +0000 UTC m=+0.503966801 container attach 8607f3d47ac52112be1aa22ea65b1e57291dd0413acaeaaf3854b52c325dc2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:33:43.898 INFO:journalctl@ceph.osd.2.smithi098.stdout:Sep 21 01:33:43 smithi098 podman[67800]: 2024-09-21 01:33:43.809109208 +0000 UTC m=+0.582188394 container died 8607f3d47ac52112be1aa22ea65b1e57291dd0413acaeaaf3854b52c325dc2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T01:33:44.512 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T01:33:44.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:44.552 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T01:33:44.552 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T01:33:44.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.3 2024-09-21T01:33:44.931 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:44 smithi027 systemd[1]: Stopping Ceph osd.3 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:45.431 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:44 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:33:44.992+0000 7f5504800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:45.431 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:44 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:33:44.992+0000 7f5504800640 -1 osd.3 102 *** Got signal Terminated *** 2024-09-21T01:33:45.431 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:44 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:33:44.992+0000 7f5504800640 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:33:46.648 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:46.235+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:47.397 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:47.248+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:47.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:47.293+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:47.143+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:47.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:47 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:47.453+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:48.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:48.296+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:48.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:48.291+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:48.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:48.110+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:48.698 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:48 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:48.432+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:49.181 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:48 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:33:48.697+0000 7f54fb200640 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:23.970898+0000 front 2024-09-21T01:33:23.970988+0000 (oldest deadline 2024-09-21T01:33:48.070951+0000) 2024-09-21T01:33:49.397 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:49.307+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:49.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:49.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:49.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:49.681 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:49 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:49.407+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:50.181 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:49 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3[53441]: 2024-09-21T01:33:49.732+0000 7f54fb200640 -1 osd.3 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:23.970898+0000 front 2024-09-21T01:33:23.970988+0000 (oldest deadline 2024-09-21T01:33:48.070951+0000) 2024-09-21T01:33:50.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:50.273+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:50.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:50.339+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:50.128+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:50.615 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:50 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:50.421+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:50.931 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:50 smithi027 podman[95210]: 2024-09-21 01:33:50.615355254 +0000 UTC m=+5.760917204 container died 846118b2aaae783e1053639bdd3d913202e6d5da6f45bb04de54a7e8a3d8763e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:33:51.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:50 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:50.856+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:51.307 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:51 smithi027 podman[95210]: 2024-09-21 01:33:51.088330772 +0000 UTC m=+6.233892722 container cleanup 846118b2aaae783e1053639bdd3d913202e6d5da6f45bb04de54a7e8a3d8763e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T01:33:51.307 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:51 smithi027 bash[95210]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3 2024-09-21T01:33:51.397 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:51.286+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:51.129+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:51.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:51.352+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:51.681 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:51 smithi027 podman[95228]: 2024-09-21 01:33:51.306364871 +0000 UTC m=+0.677233475 container remove 846118b2aaae783e1053639bdd3d913202e6d5da6f45bb04de54a7e8a3d8763e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:33:51.681 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:51 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:51.441+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:52.143 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:51 smithi027 podman[95314]: 2024-09-21 01:33:51.749897239 +0000 UTC m=+0.027795409 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:33:52.144 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:51 smithi027 podman[95314]: 2024-09-21 01:33:51.882553472 +0000 UTC m=+0.160451641 container create 2a8a25f42b6becd4ebe9994571d412cf9e81571cccc80964922fe82b19ff46f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T01:33:52.144 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:52 smithi027 podman[95314]: 2024-09-21 01:33:52.143765589 +0000 UTC m=+0.421663758 container init 2a8a25f42b6becd4ebe9994571d412cf9e81571cccc80964922fe82b19ff46f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:33:52.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:51 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:51.887+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:52.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:52.293+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:52.142+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:52.399 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:52.382+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:52.422 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:52 smithi027 podman[95314]: 2024-09-21 01:33:52.150770505 +0000 UTC m=+0.428668675 container start 2a8a25f42b6becd4ebe9994571d412cf9e81571cccc80964922fe82b19ff46f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T01:33:52.422 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:52 smithi027 podman[95314]: 2024-09-21 01:33:52.260179932 +0000 UTC m=+0.538078101 container attach 2a8a25f42b6becd4ebe9994571d412cf9e81571cccc80964922fe82b19ff46f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T01:33:52.681 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:52 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:52.490+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:52.681 INFO:journalctl@ceph.osd.3.smithi027.stdout:Sep 21 01:33:52 smithi027 podman[95337]: 2024-09-21 01:33:52.463762127 +0000 UTC m=+0.026903353 container died 2a8a25f42b6becd4ebe9994571d412cf9e81571cccc80964922fe82b19ff46f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:33:53.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T01:33:53.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:52 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:52.841+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:53.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:33:53.254 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T01:33:53.254 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T01:33:53.255 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.4 2024-09-21T01:33:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:53.122+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:53.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:53.251+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:53.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:53.357+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:53.573 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:53 smithi089 systemd[1]: Stopping Ceph osd.4 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:33:53.826 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:53 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:33:53.572+0000 7f939db4c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:33:53.826 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:53 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:33:53.572+0000 7f939db4c640 -1 osd.4 102 *** Got signal Terminated *** 2024-09-21T01:33:53.827 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:53 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:33:53.572+0000 7f939db4c640 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:33:53.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:53 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:53.460+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:54.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:53 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:53.826+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:54.396 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:54.216+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:54.396 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:54.106+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:54.647 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:54.395+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:54.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:54 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:54.503+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:55.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:54 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:54.786+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:55.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:55.108+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:55.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:55.195+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:55.898 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:55.420+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:55.898 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:55.420+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:33:55.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:55 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:55.496+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:55.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:55 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:55.496+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:33:55.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:33:55 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:33:55.526+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:33:56.000 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:55 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:33:55.790+0000 7f939612f640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.815003+0000 front 2024-09-21T01:33:29.815049+0000 (oldest deadline 2024-09-21T01:33:55.115131+0000) 2024-09-21T01:33:56.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:55 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:55.736+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:56.384 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:56 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:56.186+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:56.385 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:56 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:56.059+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:56.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:56 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:56.383+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:56.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:56 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:56.383+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:33:56.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:33:56 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:33:56.485+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:33:56.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:56 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:56.545+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:56.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:56 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:56.545+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:33:57.250 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:56 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:33:56.819+0000 7f939612f640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.815003+0000 front 2024-09-21T01:33:29.815049+0000 (oldest deadline 2024-09-21T01:33:55.115131+0000) 2024-09-21T01:33:57.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:56 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:56.786+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:57.335 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:57.164+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:57.335 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:57.011+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:57.335 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:57.011+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:33:57.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:57.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:57.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:57 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:57.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:33:57.835 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:33:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:33:57.525+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:33:57.835 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:57.771+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:57.835 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:57.771+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:33:57.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:33:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:33:57.534+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:33:57.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:57.583+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:57.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:57 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:57.583+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:33:58.250 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:57 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4[45178]: 2024-09-21T01:33:57.835+0000 7f939612f640 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.815003+0000 front 2024-09-21T01:33:29.815049+0000 (oldest deadline 2024-09-21T01:33:55.115131+0000) 2024-09-21T01:33:58.365 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:58 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:58.197+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:58.365 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:58 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:58.197+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:33:58.366 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:58 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:58.040+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:58.366 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:58 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:58.040+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:33:58.647 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:58 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:58.363+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:58.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:58 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:58.363+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:33:58.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:33:58 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:33:58.499+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:33:58.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:33:58 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:33:58.558+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:33:58.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:58 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:58.604+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:58.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:58 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:58.604+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:33:59.130 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:58 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:58.808+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:33:59.130 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:58 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:58.808+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:33:59.323 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:59 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:59.197+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:33:59.323 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:33:59 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:33:59.197+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:33:59.323 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:59 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:59.040+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:33:59.323 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:33:59 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:33:59.040+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:33:59.440 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:59 smithi089 podman[67483]: 2024-09-21 01:33:59.129695371 +0000 UTC m=+5.681296981 container died 4829958b29f3f6ddca6adbcf4c99d8c0bd3dc7d03df62ad49709fd1a26ec52f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T01:33:59.647 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:59 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:59.321+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:33:59.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:33:59 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:33:59.321+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:33:59.749 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:33:59 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:33:59.491+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:33:59.750 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:59 smithi089 podman[67483]: 2024-09-21 01:33:59.539118877 +0000 UTC m=+6.090720479 container cleanup 4829958b29f3f6ddca6adbcf4c99d8c0bd3dc7d03df62ad49709fd1a26ec52f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T01:33:59.750 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:59 smithi089 bash[67483]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4 2024-09-21T01:33:59.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:33:59 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:33:59.567+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:33:59.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:59 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:59.594+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:33:59.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:33:59 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:33:59.594+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:00.096 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:59 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:59.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:00.096 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:33:59 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:33:59.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:00.096 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:33:59 smithi089 podman[67498]: 2024-09-21 01:33:59.782296319 +0000 UTC m=+0.653598568 container remove 4829958b29f3f6ddca6adbcf4c99d8c0bd3dc7d03df62ad49709fd1a26ec52f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T01:34:00.363 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:00.194+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:00.363 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:00.194+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:00.364 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:00.037+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:00.364 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:00.037+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:00.386 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:34:00 smithi089 podman[67585]: 2024-09-21 01:33:59.998952733 +0000 UTC m=+0.015944383 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:00.386 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:34:00 smithi089 podman[67585]: 2024-09-21 01:34:00.12523733 +0000 UTC m=+0.142229020 container create 0ad7c839c1e343f7501e0a108eec1ab0e5333c41fa34f673c084bed77751e01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T01:34:00.386 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:34:00 smithi089 podman[67585]: 2024-09-21 01:34:00.386200161 +0000 UTC m=+0.403191810 container init 0ad7c839c1e343f7501e0a108eec1ab0e5333c41fa34f673c084bed77751e01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T01:34:00.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:00.362+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:00.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:00.362+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:00.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:00 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:00.454+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:00.750 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:34:00 smithi089 podman[67585]: 2024-09-21 01:34:00.389637724 +0000 UTC m=+0.406629366 container start 0ad7c839c1e343f7501e0a108eec1ab0e5333c41fa34f673c084bed77751e01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:00.750 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:34:00 smithi089 podman[67585]: 2024-09-21 01:34:00.486152127 +0000 UTC m=+0.503143770 container attach 0ad7c839c1e343f7501e0a108eec1ab0e5333c41fa34f673c084bed77751e01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:34:00.750 INFO:journalctl@ceph.osd.4.smithi089.stdout:Sep 21 01:34:00 smithi089 podman[67585]: 2024-09-21 01:34:00.575479861 +0000 UTC m=+0.592471504 container died 0ad7c839c1e343f7501e0a108eec1ab0e5333c41fa34f673c084bed77751e01d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:34:00.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:00.530+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:00.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:00.603+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:00.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:00 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:00.603+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:01.248 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:00 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:00.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:01.248 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:00 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:00.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:01.276 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T01:34:01.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:01.315 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T01:34:01.315 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T01:34:01.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.5 2024-09-21T01:34:01.343 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:01.225+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:01.343 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:01.225+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:01.343 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:00.994+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:01.343 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:00 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:00.994+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:01.499 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:01 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:01.405+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:01.639 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:01 smithi098 systemd[1]: Stopping Ceph osd.5 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:01.639 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:01.408+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:01.639 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:01.408+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:01.898 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:01.638+0000 7f35a0f9b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:01.898 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:01.638+0000 7f35a0f9b640 -1 osd.5 102 *** Got signal Terminated *** 2024-09-21T01:34:01.898 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:01 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:01.638+0000 7f35a0f9b640 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:01.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:01 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:01.536+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:01.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:01 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:01.614+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:01.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:01 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:01.614+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:02.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:01 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:01.797+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:02.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:01 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:01.797+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:02.275 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:02 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:02.022+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:02.276 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:02 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:02.022+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:02.648 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:02 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:02.273+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:02.648 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:02 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:02.273+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:02.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:02 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:02.369+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:02.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:02 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:02.369+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:02.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:02 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:02.371+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:02.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:02 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:02.489+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:02.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:02 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:02.650+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:02.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:02 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:02.650+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:03.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:02 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:02.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:03.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:02 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:02.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:03.324 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:03.281+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:03.324 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:03.281+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:03.325 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:03.069+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:03.325 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:03.069+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:03.325 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:03.069+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:03.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:03.323+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:03.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:03.323+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:03.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:03 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:03.323+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:03.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:03 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:03.399+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:03.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:03.481+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:03.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:03.696+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:03.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:03 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:03.696+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:04.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:03 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:03.871+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:04.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:03 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:03.871+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:04.305 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:04.249+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:04.305 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:04.249+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:04.305 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:04.249+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.022447+0000 front 2024-09-21T01:33:41.022355+0000 (oldest deadline 2024-09-21T01:34:03.322405+0000) 2024-09-21T01:34:04.306 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:04.045+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:04.306 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:04.045+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:04.306 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:04.045+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:04.647 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:04.305+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:04.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:04.305+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:04.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:04 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:04.305+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:04.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:04 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:04.391+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:04.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:04 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:04.481+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:04.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:04 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:04.481+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:04.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:04 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:04.673+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:04.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:04 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:04.673+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:04.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:04 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:04.673+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:05.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:04 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:04.833+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:05.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:04 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:04.833+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:05.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:05.271+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:05.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:05.271+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:05.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:05.271+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.022447+0000 front 2024-09-21T01:33:41.022355+0000 (oldest deadline 2024-09-21T01:34:03.322405+0000) 2024-09-21T01:34:05.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:05.075+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:05.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:05.075+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:05.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:05.075+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:05.399 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:05.306+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:05.399 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:05.306+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:05.399 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:05 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:05.306+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:05.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:05 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:05.410+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:05.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:05 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:05.410+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:05.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:05 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:05.513+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:05.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:05 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:05.513+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:05.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:05 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:05.637+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:05.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:05 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:05.637+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:05.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:05 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:05.637+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:06.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:05 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:05.809+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:06.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:05 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:05.809+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:06.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:06.271+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.719988+0000 front 2024-09-21T01:33:24.719867+0000 (oldest deadline 2024-09-21T01:33:45.819720+0000) 2024-09-21T01:34:06.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:06.271+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:31.720990+0000 front 2024-09-21T01:33:31.721105+0000 (oldest deadline 2024-09-21T01:33:57.620980+0000) 2024-09-21T01:34:06.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5[46102]: 2024-09-21T01:34:06.271+0000 7f359d586640 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.022447+0000 front 2024-09-21T01:33:41.022355+0000 (oldest deadline 2024-09-21T01:34:03.322405+0000) 2024-09-21T01:34:06.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:06.333+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:06.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:06.333+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:06.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:06.333+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:06.399 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:06.093+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:06.399 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:06.093+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:06.399 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:06 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:06.093+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:06.517 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:06 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:06.382+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:06.517 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:06 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:06.382+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:06.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:06.514+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:06.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:06.514+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:06.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:06.686+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:06.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:06.686+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:06.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:06.686+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:07.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:06 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:06.767+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:07.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:06 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:06.767+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:07.398 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:07 smithi098 podman[67920]: 2024-09-21 01:34:07.22491884 +0000 UTC m=+5.711737282 container died 4487c12bfc085c26630d432fa5e15fe840a26a6725d2a7ac8d33809dc138a6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:34:07.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:07 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:07.308+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:07.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:07 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:07.308+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:07.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:07 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:07.309+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:07.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:07 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:07.132+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:07.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:07 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:07.132+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:07.399 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:07 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:07.132+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:07.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:07 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:07.387+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:07.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:07 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:07.387+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:07.882 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:07 smithi098 podman[67920]: 2024-09-21 01:34:07.639119581 +0000 UTC m=+6.125938033 container cleanup 4487c12bfc085c26630d432fa5e15fe840a26a6725d2a7ac8d33809dc138a6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T01:34:07.882 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:07 smithi098 bash[67920]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5 2024-09-21T01:34:07.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:07 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:07.553+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:07.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:07 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:07.553+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:07.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:07 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:07.647+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:07.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:07 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:07.647+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:07.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:07 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:07.647+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:08.148 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:07 smithi098 podman[67935]: 2024-09-21 01:34:07.882460484 +0000 UTC m=+0.651808737 container remove 4487c12bfc085c26630d432fa5e15fe840a26a6725d2a7ac8d33809dc138a6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T01:34:08.148 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:08 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:08.122+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:08.148 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:08 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:08.122+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:08.148 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:08 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:08.122+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:08.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:07 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:07.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:08.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:07 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:07.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:08.469 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:08 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:08.295+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:08.469 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:08 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:08.295+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:08.469 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:08 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:08.295+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:08.470 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:08 smithi098 podman[68027]: 2024-09-21 01:34:08.097929436 +0000 UTC m=+0.015837682 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:08.470 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:08 smithi098 podman[68027]: 2024-09-21 01:34:08.216701554 +0000 UTC m=+0.134609788 container create 4d3abeb1969133e84c560307e1c2252c365533a958e903a7a5c4cfac98ab5254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T01:34:08.470 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:08 smithi098 podman[68027]: 2024-09-21 01:34:08.469819201 +0000 UTC m=+0.387727436 container init 4d3abeb1969133e84c560307e1c2252c365533a958e903a7a5c4cfac98ab5254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T01:34:08.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:08 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:08.363+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:08.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:08 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:08.363+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:08.898 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:08 smithi098 podman[68027]: 2024-09-21 01:34:08.473215115 +0000 UTC m=+0.391123357 container start 4d3abeb1969133e84c560307e1c2252c365533a958e903a7a5c4cfac98ab5254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:34:08.898 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:08 smithi098 podman[68027]: 2024-09-21 01:34:08.578103781 +0000 UTC m=+0.496012016 container attach 4d3abeb1969133e84c560307e1c2252c365533a958e903a7a5c4cfac98ab5254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T01:34:08.898 INFO:journalctl@ceph.osd.5.smithi098.stdout:Sep 21 01:34:08 smithi098 podman[68027]: 2024-09-21 01:34:08.6591977 +0000 UTC m=+0.577106008 container died 4d3abeb1969133e84c560307e1c2252c365533a958e903a7a5c4cfac98ab5254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T01:34:08.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:08.568+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:08.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:08.568+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:08.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:08.683+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:08.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:08.683+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:08.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:08 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:08.683+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:09.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:08 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:08.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:09.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:08 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:08.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:09.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:08 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:08.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:09.291 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:09 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:09.104+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:09.291 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:09 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:09.104+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:09.291 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:09 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:09.104+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:09.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T01:34:09.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:09.396 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T01:34:09.396 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T01:34:09.396 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.6 2024-09-21T01:34:09.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:09 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:09.322+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:09.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:09 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:09.322+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:09.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:09 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:09.322+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:09.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:09 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:09.406+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:09.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:09 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:09.406+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:09 smithi027 systemd[1]: Stopping Ceph osd.6 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:09.584+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:09.584+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:09.819+0000 7fb878e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:09.819+0000 7fb878e00640 -1 osd.6 102 *** Got signal Terminated *** 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:09.685+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:09.685+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:09.821 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:09.685+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:10.181 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:09 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:09.819+0000 7fb878e00640 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:10.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:09 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:09.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:10.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:09 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:09.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:10.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:09 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:09.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:10.397 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:10 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:10.277+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:10.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:10 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:10.277+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:10.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:10 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:10.277+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:10.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:10 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:10.099+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:10.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:10 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:10.099+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:10.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:10 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:10.099+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:10.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:10 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:10.447+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:10.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:10 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:10.447+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:10.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:10.604+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:10.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:10.604+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:10.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:10.641+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:10.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:10.641+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:10.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:10 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:10.641+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:11.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:10 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:10.753+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:11.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:10 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:10.753+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:11.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:10 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:10.753+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:11.397 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:11 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:11.251+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:11.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:11 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:11.251+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:11.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:11 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:11.251+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:11.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:11 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:11.090+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:11.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:11 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:11.090+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:11.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:11 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:11.090+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:11.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:11 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:11.449+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:11.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:11 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:11.449+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:11.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:11 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:11.449+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:11.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:11.628+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:11.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:11.628+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:11.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:11.670+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:11.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:11.670+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:11.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:11.670+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:12.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:11 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:11.770+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:12.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:11 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:11.770+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:12.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:11 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:11.770+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:12.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:12.134+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:12.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:12.134+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:12.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:12.134+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:12.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:12.222+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:12.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:12.222+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:12.398 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:12.222+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:12.399 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:12 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:12.222+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:12.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:12.490+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:12.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:12.490+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:12.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:12.490+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:12.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:12.615+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:12.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:12.615+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:12.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:12.702+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:12.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:12.702+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:12.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:12 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:12.702+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:13.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:12.754+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:13.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:12.754+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:13.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:12 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:12.754+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:13.178+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:13.178+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:13.178+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:13.266+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:13.266+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:13.266+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:13.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:13 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:13.266+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:13.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:13.442+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:13.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:13.442+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:13.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:13.442+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:13.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:13 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:13.585+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:13.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:13 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:13.585+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:13.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:13 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:13.728+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:13.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:13 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:13.728+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:13.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:13 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:13.728+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:14.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:13.749+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:14.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:13.749+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:14.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:13.749+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:14.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:13 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:13.749+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:14.221+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:14.221+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:14.221+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:14.267+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:14.267+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:14.267+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:14.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:14 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:14.267+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:14.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:14.398+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:14.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:14.398+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:14.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:14.398+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:14.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:14.545+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:29.989051+0000 front 2024-09-21T01:33:29.989017+0000 (oldest deadline 2024-09-21T01:33:55.289181+0000) 2024-09-21T01:34:14.932 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:14.545+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:41.591435+0000 front 2024-09-21T01:33:41.591582+0000 (oldest deadline 2024-09-21T01:34:03.891638+0000) 2024-09-21T01:34:14.932 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6[59915]: 2024-09-21T01:34:14.545+0000 7fb874800640 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:47.293678+0000 front 2024-09-21T01:33:47.293725+0000 (oldest deadline 2024-09-21T01:34:14.294245+0000) 2024-09-21T01:34:14.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:14.721+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:14.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:14.721+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:14.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:14.721+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:14.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:14 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:14.721+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:15.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:14.775+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:15.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:14.775+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:15.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:14.775+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:15.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:14 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:14.775+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:15.251+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:15.251+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:15.251+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:15.312+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:15.312+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:15.312+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:15.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:15 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:15.312+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:15.669 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:15 smithi027 podman[95416]: 2024-09-21 01:34:15.364092513 +0000 UTC m=+5.674491861 container died b336281eaed92a9474954f3b50bed0ae277a58def9c7d14da2ee110226d58c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T01:34:15.747 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:15.395+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:15.747 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:15.395+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:15.747 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:15.395+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:15.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:15 smithi027 podman[95416]: 2024-09-21 01:34:15.79190826 +0000 UTC m=+6.102307607 container cleanup b336281eaed92a9474954f3b50bed0ae277a58def9c7d14da2ee110226d58c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T01:34:15.931 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:15 smithi027 bash[95416]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6 2024-09-21T01:34:15.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:15 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:15.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:15.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:15 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:15.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:15.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:15 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:15.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:15.932 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:15 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:15.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:16.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:15.746+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:16.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:15.746+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:16.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:15.746+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:16.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:15.746+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:16.431 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:16 smithi027 podman[95444]: 2024-09-21 01:34:16.068451662 +0000 UTC m=+0.695154383 container remove b336281eaed92a9474954f3b50bed0ae277a58def9c7d14da2ee110226d58c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:16.205+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:16.205+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:16.205+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:16.280+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:16.280+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:16.280+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:16.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:16 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:16.280+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:16.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:16.404+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:16.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:16.404+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:16.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:16.404+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:16.905 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:16 smithi027 podman[95530]: 2024-09-21 01:34:16.518153548 +0000 UTC m=+0.028367958 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:16.905 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:16 smithi027 podman[95530]: 2024-09-21 01:34:16.644343293 +0000 UTC m=+0.154557683 container create d446c3955321a8e2ed6441fbc7ffb75792878e56be23a29a6d48d6220f9186ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T01:34:16.905 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:16 smithi027 podman[95530]: 2024-09-21 01:34:16.905093025 +0000 UTC m=+0.415307445 container init d446c3955321a8e2ed6441fbc7ffb75792878e56be23a29a6d48d6220f9186ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:34:16.905 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:16 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:16.679+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:16.905 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:16 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:16.679+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:16.905 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:16 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:16.679+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:16.906 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:16 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:16.679+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:17.181 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:16 smithi027 podman[95530]: 2024-09-21 01:34:16.912303715 +0000 UTC m=+0.422518106 container start d446c3955321a8e2ed6441fbc7ffb75792878e56be23a29a6d48d6220f9186ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:17.181 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:17 smithi027 podman[95530]: 2024-09-21 01:34:17.023646582 +0000 UTC m=+0.533860995 container attach d446c3955321a8e2ed6441fbc7ffb75792878e56be23a29a6d48d6220f9186ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:34:17.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:16.780+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:17.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:16.780+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:17.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:16.780+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:17.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:16 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:16.780+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:17.566 INFO:journalctl@ceph.osd.6.smithi027.stdout:Sep 21 01:34:17 smithi027 podman[95530]: 2024-09-21 01:34:17.183885795 +0000 UTC m=+0.694100196 container died d446c3955321a8e2ed6441fbc7ffb75792878e56be23a29a6d48d6220f9186ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:17.204+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:17.204+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:17.204+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:17.284+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:17.284+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:17.284+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:17.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:17 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:17.284+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:17.732 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:17.433+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:17.732 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:17.433+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:17.732 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:17.433+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:17.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:17.651+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:17.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:17.651+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:17.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:17.651+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:17.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:17 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:17.651+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:18.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:17.731+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:18.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:17.731+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:18.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:17.731+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:18.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:17 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:17.731+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:18.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T01:34:18.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:18.176 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T01:34:18.177 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T01:34:18.177 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.7 2024-09-21T01:34:18.454 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 systemd[1]: Stopping Ceph osd.7 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:18.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:18.201+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:18.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:18.201+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:18.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:18.201+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:18.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:18.269+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:18.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:18.269+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:18.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:18.269+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:18.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:18 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:18.269+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:18.708 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:18.454+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:18.708 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:18.454+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:18.708 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:18.454+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:18.708 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:18.500+0000 7fe6a0b64640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:18.708 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:18.500+0000 7fe6a0b64640 -1 osd.7 102 *** Got signal Terminated *** 2024-09-21T01:34:18.708 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:18.500+0000 7fe6a0b64640 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:18.999 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:18.707+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:19.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:18.707+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:19.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:18.707+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:19.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:18 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:18.707+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:19.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:18 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:18.695+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:19.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:18 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:18.695+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:19.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:18 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:18.695+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:19.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:18 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:18.695+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:19.549 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:19.175+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:19.549 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:19.175+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:19.549 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:19.175+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:19.550 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:19.253+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:19.550 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:19.253+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:19.550 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:19.253+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:19.550 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:19 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:19.253+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:19.737 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:19.451+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:19.737 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:19.451+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:19.737 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:19.451+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:19.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:19 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:19.664+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:19.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:19 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:19.664+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:19.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:19 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:19.664+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:19.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:19 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:19.664+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:20.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:19.736+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:20.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:19.736+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:20.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:19.736+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:20.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:19.736+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:20.565 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:20.225+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:20.565 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:20.225+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:20.566 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:20.225+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:20.566 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:20.283+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:20.566 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:20.283+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:20.566 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:20.283+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:20.566 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:20 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:20.283+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:20.749 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:20.480+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:20.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:20.480+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:20.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:20.480+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:20.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:20.658+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:20.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:20.658+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:20.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:20.658+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:20.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:20 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:20.658+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:21.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:20.760+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:21.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:20.760+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:21.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:20.760+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:21.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:20 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:20.760+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:21.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:21.217+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:21.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:21.217+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:21.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:21.217+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:21.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:21.307+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:21.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:21.307+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:21.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:21.307+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:21.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:21 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:21.307+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:21.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:21.448+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:21.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:21.448+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:21.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:21.448+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:21.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:21.448+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:57.317501+0000 front 2024-09-21T01:33:57.317528+0000 (oldest deadline 2024-09-21T01:34:20.817568+0000) 2024-09-21T01:34:22.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:21 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:21.698+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:22.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:21 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:21.698+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:22.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:21 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:21.698+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:22.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:21 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:21.698+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:22.182 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:21 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:21.698+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:22.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:21.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:22.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:21.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:22.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:21.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:22.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:21 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:21.796+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:22.253+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:22.253+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:22.253+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:22.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:22.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:22.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:22.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:22 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:22.334+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:22.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:22.462+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:22.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:22.462+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:22.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:22.462+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:22.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:22.462+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:57.317501+0000 front 2024-09-21T01:33:57.317528+0000 (oldest deadline 2024-09-21T01:34:20.817568+0000) 2024-09-21T01:34:23.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:22.737+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:23.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:22.737+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:23.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:22.737+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:23.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:22.737+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:23.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:22 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:22.737+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:23.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:22.825+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:23.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:22.825+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:23.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:22.825+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:23.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:22.825+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:23.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:22.825+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:23.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:23.259+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:23.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:23.259+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:23.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:23.259+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:23.649 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:23.318+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:23.649 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:23.318+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:23.649 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:23.318+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:23.649 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:23 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:23.318+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:23.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:23.421+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:33.413724+0000 front 2024-09-21T01:33:33.413756+0000 (oldest deadline 2024-09-21T01:33:56.913879+0000) 2024-09-21T01:34:23.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:23.421+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.615016+0000 front 2024-09-21T01:33:38.615150+0000 (oldest deadline 2024-09-21T01:34:04.514968+0000) 2024-09-21T01:34:23.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:23.421+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.616449+0000 front 2024-09-21T01:33:48.616330+0000 (oldest deadline 2024-09-21T01:34:10.916229+0000) 2024-09-21T01:34:23.750 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7[51804]: 2024-09-21T01:34:23.421+0000 7fe699147640 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:57.317501+0000 front 2024-09-21T01:33:57.317528+0000 (oldest deadline 2024-09-21T01:34:20.817568+0000) 2024-09-21T01:34:24.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:23 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:23.709+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:24.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:23 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:23.709+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:24.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:23 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:23.709+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:24.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:23 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:23.709+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:24.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:23 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:23.709+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:24.250 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:24 smithi089 podman[67700]: 2024-09-21 01:34:24.025241306 +0000 UTC m=+5.646105151 container died ab421179735ece1f80103aede35136fb68c877d0b03cdecb16dbdb58957b04db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:34:24.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:23.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:24.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:23.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:24.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:23.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:24.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:23.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:24.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:23 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:23.816+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:24.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:24.264+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:24.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:24.264+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:24.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:24.264+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:24.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:24.368+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:24.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:24.368+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:24.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:24.368+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:24.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:24 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:24.368+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:24.660 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:24 smithi089 podman[67700]: 2024-09-21 01:34:24.400687528 +0000 UTC m=+6.021551370 container cleanup ab421179735ece1f80103aede35136fb68c877d0b03cdecb16dbdb58957b04db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T01:34:24.660 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:24 smithi089 bash[67700]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7 2024-09-21T01:34:25.000 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:24 smithi089 podman[67716]: 2024-09-21 01:34:24.660206379 +0000 UTC m=+0.635137546 container remove ab421179735ece1f80103aede35136fb68c877d0b03cdecb16dbdb58957b04db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T01:34:25.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:24 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:24.783+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:25.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:24 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:24.783+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:25.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:24 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:24.783+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:25.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:24 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:24.783+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:25.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:24 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:24.783+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:25.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:24 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:24.710+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:25.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:24 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:24.710+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:25.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:24 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:24.710+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:25.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:24 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:24.710+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:25.182 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:24 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:24.710+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:25.263 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:25 smithi089 podman[67802]: 2024-09-21 01:34:24.915865269 +0000 UTC m=+0.015692937 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:25.264 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:25 smithi089 podman[67802]: 2024-09-21 01:34:25.019321929 +0000 UTC m=+0.119149586 container create 59e343b7d5a8b8f1def118c59bb0aec99eaa7386bc90f2007ccbaed5639ec86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T01:34:25.264 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:25 smithi089 podman[67802]: 2024-09-21 01:34:25.263783683 +0000 UTC m=+0.363611352 container init 59e343b7d5a8b8f1def118c59bb0aec99eaa7386bc90f2007ccbaed5639ec86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T01:34:25.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:25.229+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:25.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:25.229+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:25.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:25.229+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:25.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:25.229+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:25.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:25.377+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:25.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:25.377+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:25.649 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:25.377+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:25.649 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:25 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:25.377+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:25.745 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:25 smithi089 podman[67802]: 2024-09-21 01:34:25.267299749 +0000 UTC m=+0.367127422 container start 59e343b7d5a8b8f1def118c59bb0aec99eaa7386bc90f2007ccbaed5639ec86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T01:34:25.745 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:25 smithi089 podman[67802]: 2024-09-21 01:34:25.347013757 +0000 UTC m=+0.446841415 container attach 59e343b7d5a8b8f1def118c59bb0aec99eaa7386bc90f2007ccbaed5639ec86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T01:34:25.745 INFO:journalctl@ceph.osd.7.smithi089.stdout:Sep 21 01:34:25 smithi089 podman[67802]: 2024-09-21 01:34:25.456512401 +0000 UTC m=+0.556340065 container died 59e343b7d5a8b8f1def118c59bb0aec99eaa7386bc90f2007ccbaed5639ec86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T01:34:25.999 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:25.744+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:26.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:25.744+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:26.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:25.744+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:26.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:25.744+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:26.000 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:25 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:25.744+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:26.132 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T01:34:26.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:26.173 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T01:34:26.173 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T01:34:26.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.8 2024-09-21T01:34:26.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:25 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:25.740+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:26.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:25 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:25.740+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:26.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:25 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:25.740+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:26.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:25 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:25.740+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:26.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:25 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:25.740+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:26.494 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:26.215+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:26.494 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:26.216+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:26.494 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:26.216+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:26.494 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:26.216+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 systemd[1]: Stopping Ceph osd.8 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.382+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.382+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.382+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.382+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.493+0000 7f71ceaf1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.493+0000 7f71ceaf1640 -1 osd.8 102 *** Got signal Terminated *** 2024-09-21T01:34:26.495 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:26 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:26.493+0000 7f71ceaf1640 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:27.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:26.770+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:27.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:26.770+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:27.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:26.770+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:27.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:26.770+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:27.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:26 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:26.770+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:27.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:26.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:27.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:26.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:27.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:26.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:27.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:26.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:27.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:26.777+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:27.218+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:27.218+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:27.218+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:27.218+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:27.341+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:27.341+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:27.341+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:27.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:27 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:27.341+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:28.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:27 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:27.789+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:28.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:27 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:27.789+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:28.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:27 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:27.789+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:28.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:27 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:27.789+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:28.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:27 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:27.789+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:28.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:27 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:27.821+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:28.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:27 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:27.821+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:28.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:27 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:27.821+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:28.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:27 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:27.821+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:28.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:27 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:27.821+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:28.263+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:28.263+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:28.263+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:28.263+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:28.302+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:28.302+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:28.302+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:28.302+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:28.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:28 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:28.302+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:04.209491+0000 front 2024-09-21T01:34:04.209546+0000 (oldest deadline 2024-09-21T01:34:27.709562+0000) 2024-09-21T01:34:29.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:28 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:28.826+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:29.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:28 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:28.826+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:29.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:28 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:28.826+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:29.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:28 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:28.826+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:29.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:28 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:28.826+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:29.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:28.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:29.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:28.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:29.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:28.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:29.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:28.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:29.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:28.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:29.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:28 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:28.784+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:29.254+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:29.254+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:29.254+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:29.254+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:29.254+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:04.209491+0000 front 2024-09-21T01:34:04.209546+0000 (oldest deadline 2024-09-21T01:34:27.709562+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:29.297+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:29.297+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:29.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:29.297+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:29.649 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:29 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:29.297+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:30.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:29 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:29.810+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:30.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:29 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:29.810+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:30.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:29 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:29.810+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:30.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:29 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:29.810+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:30.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:29 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:29.810+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:30.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:29 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:29.766+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:30.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:29 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:29.766+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:30.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:29 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:29.766+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:30.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:29 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:29.766+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:30.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:29 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:29.766+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:30.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:29 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:29.766+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:30.244+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:30.244+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:30.244+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:30.244+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:30.244+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:04.209491+0000 front 2024-09-21T01:34:04.209546+0000 (oldest deadline 2024-09-21T01:34:27.709562+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:30.303+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:30.303+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:30.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:30.303+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:30.649 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:30.303+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:31.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:30 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:30.796+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:31.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:30 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:30.796+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:31.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:30 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:30.796+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:31.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:30 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:30.796+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:31.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:30 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:30.796+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:31.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:30 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:30.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:31.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:30 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:30.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:31.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:30 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:30.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:31.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:30 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:30.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:31.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:30 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:30.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:31.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:30 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:30.812+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:31.237+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.601098+0000 front 2024-09-21T01:33:22.601189+0000 (oldest deadline 2024-09-21T01:33:46.701005+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:31.237+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:28.302708+0000 front 2024-09-21T01:33:28.302853+0000 (oldest deadline 2024-09-21T01:33:54.703459+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:31.237+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:38.104785+0000 front 2024-09-21T01:33:38.104918+0000 (oldest deadline 2024-09-21T01:34:02.804953+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:31.237+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.706216+0000 front 2024-09-21T01:33:48.706163+0000 (oldest deadline 2024-09-21T01:34:12.206134+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8[52616]: 2024-09-21T01:34:31.237+0000 7f71cb0dc640 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:04.209491+0000 front 2024-09-21T01:34:04.209546+0000 (oldest deadline 2024-09-21T01:34:27.709562+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:31.336+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:31.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:31.336+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:31.649 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:31.336+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:31.649 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:31.336+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:31.649 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:31 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:31.336+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:32.148 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:31 smithi098 podman[68142]: 2024-09-21 01:34:31.90216692 +0000 UTC m=+5.534121989 container died bbd3032f6b87e968424e1fdadf3453eb73b6a6a5a9105605e3d10cfa1ac4aba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:34:32.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:31 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:31.783+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:32.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:31 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:31.784+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:32.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:31 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:31.784+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:32.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:31 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:31.784+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:32.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:31 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:31.784+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:32.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:31 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:31.784+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:32.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:31 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:31.840+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:32.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:31 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:31.840+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:32.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:31 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:31.840+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:32.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:31 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:31.840+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:32.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:31 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:31.840+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:32.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:31 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:31.840+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:32.587 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:32 smithi098 podman[68142]: 2024-09-21 01:34:32.310944633 +0000 UTC m=+5.942899711 container cleanup bbd3032f6b87e968424e1fdadf3453eb73b6a6a5a9105605e3d10cfa1ac4aba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:34:32.587 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:32 smithi098 bash[68142]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8 2024-09-21T01:34:32.587 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:32 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:32.340+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:32.587 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:32 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:32.340+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:32.587 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:32 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:32.340+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:32.587 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:32 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:32.340+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:32.588 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:32 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:32.340+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:32.898 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:32 smithi098 podman[68159]: 2024-09-21 01:34:32.587429498 +0000 UTC m=+0.681391583 container remove bbd3032f6b87e968424e1fdadf3453eb73b6a6a5a9105605e3d10cfa1ac4aba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T01:34:33.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:32 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:32.734+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:33.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:32 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:32.734+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:33.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:32 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:32.734+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:33.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:32 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:32.734+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:33.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:32 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:32.734+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:33.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:32 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:32.734+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:33.249 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:32 smithi098 podman[68250]: 2024-09-21 01:34:32.820945317 +0000 UTC m=+0.015267088 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:33.249 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:32 smithi098 podman[68250]: 2024-09-21 01:34:32.929914311 +0000 UTC m=+0.124236066 container create ae8faf535a723ef1b7fd02f5d6a215d97a81d870e59e438c6fbfb5e0c299d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T01:34:33.249 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:33 smithi098 podman[68250]: 2024-09-21 01:34:33.157601432 +0000 UTC m=+0.351923186 container init ae8faf535a723ef1b7fd02f5d6a215d97a81d870e59e438c6fbfb5e0c299d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:34:33.249 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:33 smithi098 podman[68250]: 2024-09-21 01:34:33.161064109 +0000 UTC m=+0.355385862 container start ae8faf535a723ef1b7fd02f5d6a215d97a81d870e59e438c6fbfb5e0c299d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:33 smithi098 podman[68250]: 2024-09-21 01:34:33.249352184 +0000 UTC m=+0.443673937 container attach ae8faf535a723ef1b7fd02f5d6a215d97a81d870e59e438c6fbfb5e0c299d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:32 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:32.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:32 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:32.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:32 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:32.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:32 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:32.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:32 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:32.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:33.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:32 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:32.845+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:33.648 INFO:journalctl@ceph.osd.8.smithi098.stdout:Sep 21 01:34:33 smithi098 podman[68250]: 2024-09-21 01:34:33.352402615 +0000 UTC m=+0.546724375 container died ae8faf535a723ef1b7fd02f5d6a215d97a81d870e59e438c6fbfb5e0c299d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:33.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:33 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:33.301+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:33.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:33 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:33.301+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:33.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:33 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:33.301+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:33.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:33 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:33.301+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:33.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:33 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:33.301+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:33.970 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T01:34:34.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:34.010 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T01:34:34.010 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T01:34:34.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.9 2024-09-21T01:34:34.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:33 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:33.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:34.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:33 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:33.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:34.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:33 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:33.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:34.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:33 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:33.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:34.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:33 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:33.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:34.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:33 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:33.705+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:34.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:33 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:33.843+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:34.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:33 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:33.843+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:34.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:33 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:33.843+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:34.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:33 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:33.843+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:34.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:33 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:33.843+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:34.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:33 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:33.843+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:34.431 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 systemd[1]: Stopping Ceph osd.9 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:34.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:34.339+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:34.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:34.339+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:34.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:34.339+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:34.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:34.339+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:34.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:34.339+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:34.717 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.430+0000 7f0855e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:34.717 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.430+0000 7f0855e00640 -1 osd.9 102 *** Got signal Terminated *** 2024-09-21T01:34:34.717 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.430+0000 7f0855e00640 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:34.717 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.716+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:34.718 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.716+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:35.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.716+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:35.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.716+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:35.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.716+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:35.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:34 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:34.716+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:35.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:34 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:34.849+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:35.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:34 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:34.849+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:35.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:34 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:34.849+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:35.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:34 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:34.849+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:35.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:34 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:34.849+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:35.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:34 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:34.849+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:35.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:35 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:35.348+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:35.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:35 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:35.348+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:35.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:35 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:35.348+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:35.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:35 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:35.348+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:35.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:35 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:35.348+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:35.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:35 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:35.671+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:35.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:35 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:35.671+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:35.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:35 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:35.671+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:35.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:35 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:35.671+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:35.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:35 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:35.671+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:35.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:35 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:35.671+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:36.151 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:35 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:35.898+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:36.151 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:35 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:35.898+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:36.151 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:35 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:35.898+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:36.151 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:35 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:35.898+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:36.151 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:35 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:35.898+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:36.151 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:35 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:35.898+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:36.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:36.389+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:36.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:36.389+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:36.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:36.389+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:36.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:36.389+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:36.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:36.389+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:36.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:36 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:36.389+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:36.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:36 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:36.662+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:36.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:36 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:36.662+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:36.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:36 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:36.662+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:36.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:36 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:36.662+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:36.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:36 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:36.662+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:36.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:36 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:36.662+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:37.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:36 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:36.922+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:37.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:36 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:36.922+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:37.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:36 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:36.922+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:37.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:36 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:36.922+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:37.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:36 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:36.922+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:37.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:36 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:36.922+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:37.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:37 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:37.363+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:37.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:37 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:37.363+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:37.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:37 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:37.363+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:37.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:37 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:37.363+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:37.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:37 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:37.363+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:37.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:37 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:37.363+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:38.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:37 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:37.684+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:38.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:37 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:37.684+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:38.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:37 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:37.684+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:38.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:37 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:37.684+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:38.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:37 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:37.684+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:38.181 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:37 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:37.684+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:38.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:37 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:37.960+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:38.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:37 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:37.960+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:38.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:37 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:37.960+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:38.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:37 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:37.960+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:38.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:37 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:37.960+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:38.251 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:37 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:37.960+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:38.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:38.321+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:38.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:38.321+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:38.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:38.321+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:38.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:38.321+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:38.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:38.321+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:38.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:38.321+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:38.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:38 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:38.652+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:22.907702+0000 front 2024-09-21T01:33:22.907809+0000 (oldest deadline 2024-09-21T01:33:47.007768+0000) 2024-09-21T01:34:38.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:38 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:38.652+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.209940+0000 front 2024-09-21T01:33:32.210042+0000 (oldest deadline 2024-09-21T01:33:55.109861+0000) 2024-09-21T01:34:38.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:38 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:38.652+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:39.811206+0000 front 2024-09-21T01:33:39.811170+0000 (oldest deadline 2024-09-21T01:34:04.511351+0000) 2024-09-21T01:34:38.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:38 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:38.652+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:49.212324+0000 front 2024-09-21T01:33:49.212411+0000 (oldest deadline 2024-09-21T01:34:14.413013+0000) 2024-09-21T01:34:38.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:38 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:38.652+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.713913+0000 front 2024-09-21T01:33:56.714026+0000 (oldest deadline 2024-09-21T01:34:21.414112+0000) 2024-09-21T01:34:38.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:38 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9[66597]: 2024-09-21T01:34:38.652+0000 7f0851800640 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:06.115430+0000 front 2024-09-21T01:34:06.115234+0000 (oldest deadline 2024-09-21T01:34:31.415443+0000) 2024-09-21T01:34:39.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:38.967+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:39.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:38.967+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:39.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:38.967+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:39.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:38.967+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:39.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:38.967+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:39.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:38 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:38.967+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:39.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:39.276+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:39.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:39.276+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:39.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:39.276+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:39.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:39.276+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:39.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:39.276+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:39.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:39 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:39.276+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:40.145 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:39 smithi027 podman[95631]: 2024-09-21 01:34:39.859144205 +0000 UTC m=+5.545974045 container died c58f43a4a51fc22f9a41c66e3ced669c65881c1d2e90406abefd2f0d457edac1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T01:34:40.249 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:40.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:40.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:40.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:40.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:40.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:40.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:39 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:39.946+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:40.431 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:40 smithi027 podman[95631]: 2024-09-21 01:34:40.25230206 +0000 UTC m=+5.939131902 container cleanup c58f43a4a51fc22f9a41c66e3ced669c65881c1d2e90406abefd2f0d457edac1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T01:34:40.431 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:40 smithi027 bash[95631]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9 2024-09-21T01:34:40.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:40 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:40.257+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:40.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:40 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:40.257+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:40.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:40 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:40.257+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:40.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:40 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:40.257+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:40.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:40 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:40.257+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:40.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:40 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:40.257+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:40.931 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:40 smithi027 podman[95646]: 2024-09-21 01:34:40.645477937 +0000 UTC m=+0.779348811 container remove c58f43a4a51fc22f9a41c66e3ced669c65881c1d2e90406abefd2f0d457edac1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:41.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:40 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:40.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:41.431 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:41 smithi027 podman[95728]: 2024-09-21 01:34:40.994663503 +0000 UTC m=+0.028543363 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:41.431 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:41 smithi027 podman[95728]: 2024-09-21 01:34:41.104335257 +0000 UTC m=+0.138215117 container create 42b5f71b2e67d2553320edb7dd77c91184dd0187940ca37ad5bfccfff71933d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-21T01:34:41.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:41.213+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:41.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:41.213+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:41.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:41.213+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:41.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:41.213+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:41.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:41.213+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:41.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:41 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:41.213+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:41.755 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:41 smithi027 podman[95728]: 2024-09-21 01:34:41.481988883 +0000 UTC m=+0.515868744 container init 42b5f71b2e67d2553320edb7dd77c91184dd0187940ca37ad5bfccfff71933d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:34:41.755 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:41 smithi027 podman[95728]: 2024-09-21 01:34:41.488981692 +0000 UTC m=+0.522861552 container start 42b5f71b2e67d2553320edb7dd77c91184dd0187940ca37ad5bfccfff71933d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T01:34:41.755 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:41 smithi027 podman[95728]: 2024-09-21 01:34:41.623425058 +0000 UTC m=+0.657304934 container attach 42b5f71b2e67d2553320edb7dd77c91184dd0187940ca37ad5bfccfff71933d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T01:34:42.067 INFO:journalctl@ceph.osd.9.smithi027.stdout:Sep 21 01:34:41 smithi027 podman[95751]: 2024-09-21 01:34:41.796338188 +0000 UTC m=+0.027378945 container died 42b5f71b2e67d2553320edb7dd77c91184dd0187940ca37ad5bfccfff71933d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:42.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:41 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:41.949+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:42.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T01:34:42.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:42.466 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T01:34:42.466 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T01:34:42.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.10 2024-09-21T01:34:42.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:42.172+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:42.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:42.172+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:42.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:42.172+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:42.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:42.172+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:42.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:42.172+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:42.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:42.172+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:42.749 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 systemd[1]: Stopping Ceph osd.10 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:43.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.769+0000 7fc256c74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:43.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.769+0000 7fc256c74640 -1 osd.10 102 *** Got signal Terminated *** 2024-09-21T01:34:43.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.769+0000 7fc256c74640 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:43.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:43.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:43.251 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:43.251 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:43.251 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:43.251 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:43.251 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:42 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:42.938+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:43.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:43.133+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:43.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:43.133+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:43.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:43.133+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:43.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:43.133+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:43.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:43.133+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:43.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:43 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:43.133+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:44.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:43 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:43.942+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:44.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:44 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:44.109+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:44.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:44 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:44.109+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:44.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:44 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:44.109+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:44.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:44 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:44.109+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:44.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:44 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:44.109+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:44.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:44 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:44.109+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:45.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:44 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:44.905+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:45.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:45 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:45.144+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:46.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:45 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:45.931+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:46.648 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:46 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:46.152+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:24.704317+0000 front 2024-09-21T01:33:24.704520+0000 (oldest deadline 2024-09-21T01:33:50.004483+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:32.905346+0000 front 2024-09-21T01:33:32.905299+0000 (oldest deadline 2024-09-21T01:33:57.605439+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:40.506393+0000 front 2024-09-21T01:33:40.506464+0000 (oldest deadline 2024-09-21T01:34:08.106839+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6814 osd.3 since back 2024-09-21T01:33:48.107559+0000 front 2024-09-21T01:33:48.107632+0000 (oldest deadline 2024-09-21T01:34:12.807411+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:56.908196+0000 front 2024-09-21T01:33:56.908141+0000 (oldest deadline 2024-09-21T01:34:22.108611+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:02.109136+0000 front 2024-09-21T01:34:02.109169+0000 (oldest deadline 2024-09-21T01:34:27.909591+0000) 2024-09-21T01:34:47.250 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:46 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10[58358]: 2024-09-21T01:34:46.903+0000 7fc25325f640 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.210556+0000 front 2024-09-21T01:34:13.210599+0000 (oldest deadline 2024-09-21T01:34:39.110407+0000) 2024-09-21T01:34:47.397 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:47.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:47 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:47.130+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:48.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:48.422 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:48.422 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:48.423 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:48.423 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:48.423 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:48.423 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:48 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:48.102+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:48.595 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:48 smithi089 podman[67920]: 2024-09-21 01:34:48.300368223 +0000 UTC m=+5.631004325 container died 3d88876e89a60010807242b31afd3d503f21235dae3142d2cba02a8fab2128a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T01:34:48.920 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:48 smithi089 podman[67920]: 2024-09-21 01:34:48.67766361 +0000 UTC m=+6.008299710 container cleanup 3d88876e89a60010807242b31afd3d503f21235dae3142d2cba02a8fab2128a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:48.920 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:48 smithi089 bash[67920]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10 2024-09-21T01:34:49.236 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:48 smithi089 podman[67937]: 2024-09-21 01:34:48.920325093 +0000 UTC m=+0.615117997 container remove 3d88876e89a60010807242b31afd3d503f21235dae3142d2cba02a8fab2128a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:49.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:49 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:49.145+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:49.500 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:49 smithi089 podman[68025]: 2024-09-21 01:34:49.138768782 +0000 UTC m=+0.015744430 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:49.500 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:49 smithi089 podman[68025]: 2024-09-21 01:34:49.237693621 +0000 UTC m=+0.114669253 container create fa2d42e16dd8cafe57dc9d249f475d68a018fa1d203662fec2a5aac0d0f2a78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:34:49.500 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:49 smithi089 podman[68025]: 2024-09-21 01:34:49.448565461 +0000 UTC m=+0.325541093 container init fa2d42e16dd8cafe57dc9d249f475d68a018fa1d203662fec2a5aac0d0f2a78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T01:34:49.500 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:49 smithi089 podman[68025]: 2024-09-21 01:34:49.451728896 +0000 UTC m=+0.328704541 container start fa2d42e16dd8cafe57dc9d249f475d68a018fa1d203662fec2a5aac0d0f2a78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T01:34:49.926 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:49 smithi089 podman[68025]: 2024-09-21 01:34:49.531660167 +0000 UTC m=+0.408635817 container attach fa2d42e16dd8cafe57dc9d249f475d68a018fa1d203662fec2a5aac0d0f2a78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:49.926 INFO:journalctl@ceph.osd.10.smithi089.stdout:Sep 21 01:34:49 smithi089 podman[68025]: 2024-09-21 01:34:49.640548019 +0000 UTC m=+0.517523661 container died fa2d42e16dd8cafe57dc9d249f475d68a018fa1d203662fec2a5aac0d0f2a78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T01:34:50.222 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T01:34:50.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:50.261 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T01:34:50.262 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T01:34:50.262 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.11 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:50.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:50.399 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 systemd[1]: Stopping Ceph osd.11 for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:50.897 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.581+0000 7f76e92ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:50.898 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.581+0000 7f76e92ea640 -1 osd.11 102 *** Got signal Terminated *** 2024-09-21T01:34:50.898 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:50 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:50.581+0000 7f76e92ea640 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.112+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.112+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.112+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.112+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.112+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:51.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:51 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:51.113+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.099+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.099+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.100+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.100+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.100+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.100+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:52.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:52 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:52.100+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:53.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:53 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:53.088+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:54.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:54 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:54.115+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6820 osd.8 since back 2024-09-21T01:34:30.284808+0000 front 2024-09-21T01:34:30.284851+0000 (oldest deadline 2024-09-21T01:34:53.184872+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6806 osd.0 since back 2024-09-21T01:33:21.472322+0000 front 2024-09-21T01:33:21.472198+0000 (oldest deadline 2024-09-21T01:33:46.172197+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-09-21T01:33:30.773866+0000 front 2024-09-21T01:33:30.773619+0000 (oldest deadline 2024-09-21T01:33:56.673578+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6804 osd.2 since back 2024-09-21T01:33:37.674997+0000 front 2024-09-21T01:33:37.675043+0000 (oldest deadline 2024-09-21T01:34:02.375154+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-09-21T01:33:58.379889+0000 front 2024-09-21T01:33:58.380055+0000 (oldest deadline 2024-09-21T01:34:24.279790+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6812 osd.5 since back 2024-09-21T01:34:05.380672+0000 front 2024-09-21T01:34:05.380692+0000 (oldest deadline 2024-09-21T01:34:30.680756+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.27:6822 osd.6 since back 2024-09-21T01:34:13.482652+0000 front 2024-09-21T01:34:13.482668+0000 (oldest deadline 2024-09-21T01:34:35.782225+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-09-21T01:34:20.884058+0000 front 2024-09-21T01:34:20.883973+0000 (oldest deadline 2024-09-21T01:34:44.983895+0000) 2024-09-21T01:34:55.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:55 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11[58820]: 2024-09-21T01:34:55.094+0000 7f76e18cd640 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.98:6820 osd.8 since back 2024-09-21T01:34:30.284808+0000 front 2024-09-21T01:34:30.284851+0000 (oldest deadline 2024-09-21T01:34:53.184872+0000) 2024-09-21T01:34:56.398 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:56 smithi098 podman[68374]: 2024-09-21 01:34:56.109698637 +0000 UTC m=+5.665836504 container died 74d477139946bd526b092455757fbaecaea40d8b5c4010ab64f0719f164a7f13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:56.741 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:56 smithi098 podman[68374]: 2024-09-21 01:34:56.48979857 +0000 UTC m=+6.045936440 container cleanup 74d477139946bd526b092455757fbaecaea40d8b5c4010ab64f0719f164a7f13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T01:34:56.741 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:56 smithi098 bash[68374]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11 2024-09-21T01:34:57.047 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:56 smithi098 podman[68390]: 2024-09-21 01:34:56.741058528 +0000 UTC m=+0.632772669 container remove 74d477139946bd526b092455757fbaecaea40d8b5c4010ab64f0719f164a7f13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T01:34:57.352 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:57 smithi098 podman[68479]: 2024-09-21 01:34:56.950047698 +0000 UTC m=+0.015898197 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T01:34:57.352 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:57 smithi098 podman[68479]: 2024-09-21 01:34:57.049955318 +0000 UTC m=+0.115805806 container create 17e519bef37756bac6819be9c1d8382ed6e9e358a0711426898d2bd6ca5538c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T01:34:57.352 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:57 smithi098 podman[68479]: 2024-09-21 01:34:57.269252204 +0000 UTC m=+0.335102703 container init 17e519bef37756bac6819be9c1d8382ed6e9e358a0711426898d2bd6ca5538c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T01:34:57.353 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:57 smithi098 podman[68479]: 2024-09-21 01:34:57.272524882 +0000 UTC m=+0.338375383 container start 17e519bef37756bac6819be9c1d8382ed6e9e358a0711426898d2bd6ca5538c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T01:34:57.353 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:57 smithi098 podman[68479]: 2024-09-21 01:34:57.352431967 +0000 UTC m=+0.418282481 container attach 17e519bef37756bac6819be9c1d8382ed6e9e358a0711426898d2bd6ca5538c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-21T01:34:57.647 INFO:journalctl@ceph.osd.11.smithi098.stdout:Sep 21 01:34:57 smithi098 podman[68479]: 2024-09-21 01:34:57.457899152 +0000 UTC m=+0.523749649 container died 17e519bef37756bac6819be9c1d8382ed6e9e358a0711426898d2bd6ca5538c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T01:34:58.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T01:34:58.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:34:58.079 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T01:34:58.079 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T01:34:58.079 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.a 2024-09-21T01:34:58.431 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:34:58 smithi027 systemd[1]: Stopping Ceph mds.a for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:34:58.931 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:34:58 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a[73437]: 2024-09-21T01:34:58.515+0000 7fe903000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:34:58.931 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:34:58 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a[73437]: 2024-09-21T01:34:58.515+0000 7fe903000640 -1 mds.a *** got signal Terminated *** 2024-09-21T01:35:00.181 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:34:59 smithi027 podman[95842]: 2024-09-21 01:34:59.791410193 +0000 UTC m=+1.403488527 container stop a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T01:35:00.181 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:34:59 smithi027 podman[95842]: 2024-09-21 01:34:59.817658581 +0000 UTC m=+1.429736925 container died a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T01:35:00.563 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:35:00 smithi027 podman[95842]: 2024-09-21 01:35:00.370072823 +0000 UTC m=+1.982151155 container cleanup a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T01:35:00.563 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:35:00 smithi027 bash[95842]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a 2024-09-21T01:35:00.851 INFO:journalctl@ceph.mds.a.smithi027.stdout:Sep 21 01:35:00 smithi027 podman[95854]: 2024-09-21 01:35:00.562718398 +0000 UTC m=+0.750957541 container remove a619101c83b893268ea16e08324c3713d74e94c8a6936aeae5d8a0c95354b561 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T01:35:00.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T01:35:00.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:00.944 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T01:35:00.945 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T01:35:00.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.d 2024-09-21T01:35:01.181 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:01 smithi027 systemd[1]: Stopping Ceph mds.d for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:01.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:01 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d[72322]: 2024-09-21T01:35:01.468+0000 7f62f9e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:01.931 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:01 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d[72322]: 2024-09-21T01:35:01.468+0000 7f62f9e00640 -1 mds.d *** got signal Terminated *** 2024-09-21T01:35:05.817 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:05 smithi027 podman[95962]: 2024-09-21 01:35:05.494356811 +0000 UTC m=+4.257857324 container died 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T01:35:06.101 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:05 smithi027 podman[95962]: 2024-09-21 01:35:05.931961007 +0000 UTC m=+4.695461518 container cleanup 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T01:35:06.101 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:05 smithi027 bash[95962]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d 2024-09-21T01:35:06.358 INFO:journalctl@ceph.mds.d.smithi027.stdout:Sep 21 01:35:06 smithi027 podman[95986]: 2024-09-21 01:35:06.100048676 +0000 UTC m=+0.592241827 container remove 2ac635b2c3440c1ad2dc1181c0938fe68e65d42cadf4ea4665a6f4f9f7d2f0a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T01:35:06.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T01:35:06.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:06.455 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T01:35:06.455 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T01:35:06.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.g 2024-09-21T01:35:06.681 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:06 smithi027 systemd[1]: Stopping Ceph mds.g for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:07.181 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g[73090]: 2024-09-21T01:35:06.838+0000 7f0722c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:07.181 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:06 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g[73090]: 2024-09-21T01:35:06.838+0000 7f0722c00640 -1 mds.g *** got signal Terminated *** 2024-09-21T01:35:10.181 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:09 smithi027 podman[96092]: 2024-09-21 01:35:09.86234508 +0000 UTC m=+3.129556051 container died 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T01:35:10.637 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:10 smithi027 podman[96092]: 2024-09-21 01:35:10.310694385 +0000 UTC m=+3.577905355 container cleanup 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T01:35:10.637 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:10 smithi027 bash[96092]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g 2024-09-21T01:35:10.931 INFO:journalctl@ceph.mds.g.smithi027.stdout:Sep 21 01:35:10 smithi027 podman[96104]: 2024-09-21 01:35:10.636673854 +0000 UTC m=+0.763004268 container remove 0d3e5d93f44c50c03b8cf2de6314c7157cf921c4c36b1bedc2e5117be5747bfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T01:35:10.969 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T01:35:11.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:11.041 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T01:35:11.041 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T01:35:11.041 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.j 2024-09-21T01:35:11.431 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:35:11 smithi027 systemd[1]: Stopping Ceph mds.j for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:11.931 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:35:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j[71942]: 2024-09-21T01:35:11.442+0000 7f8b26800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:11.931 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:35:11 smithi027 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j[71942]: 2024-09-21T01:35:11.442+0000 7f8b26800640 -1 mds.j *** got signal Terminated *** 2024-09-21T01:35:14.165 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:35:13 smithi027 podman[96212]: 2024-09-21 01:35:13.866253173 +0000 UTC m=+2.533982668 container died 612eec67dae253f57a8f13bea185d3941836dc7fc4f808d676f0f17e0cec7726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-21T01:35:14.431 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:35:14 smithi027 podman[96212]: 2024-09-21 01:35:14.272331277 +0000 UTC m=+2.940060772 container cleanup 612eec67dae253f57a8f13bea185d3941836dc7fc4f808d676f0f17e0cec7726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:35:14.431 INFO:journalctl@ceph.mds.j.smithi027.stdout:Sep 21 01:35:14 smithi027 bash[96212]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-j 2024-09-21T01:35:14.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T01:35:14.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:14.771 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T01:35:14.771 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T01:35:14.771 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.b 2024-09-21T01:35:15.119 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:35:14 smithi089 systemd[1]: Stopping Ceph mds.b for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:15.499 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:35:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: 2024-09-21T01:35:15.119+0000 7f00afdb0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:15.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:35:15 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b[63401]: 2024-09-21T01:35:15.119+0000 7f00afdb0640 -1 mds.b *** got signal Terminated *** 2024-09-21T01:35:18.000 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:35:17 smithi089 podman[68127]: 2024-09-21 01:35:17.729194442 +0000 UTC m=+2.716569762 container died 2867b76e2e4f7814892dd50edf84c90166bce9f24b95ba8b1ef00df7c8e13bc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T01:35:18.499 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:35:18 smithi089 podman[68127]: 2024-09-21 01:35:18.293605338 +0000 UTC m=+3.280980667 container cleanup 2867b76e2e4f7814892dd50edf84c90166bce9f24b95ba8b1ef00df7c8e13bc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T01:35:18.500 INFO:journalctl@ceph.mds.b.smithi089.stdout:Sep 21 01:35:18 smithi089 bash[68127]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-b 2024-09-21T01:35:18.773 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T01:35:18.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:18.809 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T01:35:18.809 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T01:35:18.810 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.e 2024-09-21T01:35:19.123 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:35:18 smithi089 systemd[1]: Stopping Ceph mds.e for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:19.500 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:35:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e[62857]: 2024-09-21T01:35:19.122+0000 7efcc2356640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:19.500 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:35:19 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e[62857]: 2024-09-21T01:35:19.122+0000 7efcc2356640 -1 mds.e *** got signal Terminated *** 2024-09-21T01:35:22.000 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:35:21 smithi089 podman[68259]: 2024-09-21 01:35:21.733578142 +0000 UTC m=+2.711701625 container died eac84a7a3c52ad490bf3cd859f282bc1c644e975188ca888ec38ba30cfb686d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:35:22.381 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:35:22 smithi089 podman[68259]: 2024-09-21 01:35:22.097424384 +0000 UTC m=+3.075547876 container cleanup eac84a7a3c52ad490bf3cd859f282bc1c644e975188ca888ec38ba30cfb686d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T01:35:22.382 INFO:journalctl@ceph.mds.e.smithi089.stdout:Sep 21 01:35:22 smithi089 bash[68259]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-e 2024-09-21T01:35:22.535 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T01:35:22.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:22.568 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T01:35:22.568 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T01:35:22.568 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.h 2024-09-21T01:35:22.999 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:35:22 smithi089 systemd[1]: Stopping Ceph mds.h for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:23.000 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:35:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h[63132]: 2024-09-21T01:35:22.876+0000 7fb452019640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:23.000 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:35:22 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h[63132]: 2024-09-21T01:35:22.876+0000 7fb452019640 -1 mds.h *** got signal Terminated *** 2024-09-21T01:35:26.000 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:35:25 smithi089 podman[68380]: 2024-09-21 01:35:25.734036258 +0000 UTC m=+2.951065214 container died 87025dbafe51c00abc38712e666ced2e38e3a56f2307c378f9edc6d984b6c7c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T01:35:26.376 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:35:26 smithi089 podman[68380]: 2024-09-21 01:35:26.084096125 +0000 UTC m=+3.301125083 container cleanup 87025dbafe51c00abc38712e666ced2e38e3a56f2307c378f9edc6d984b6c7c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:35:26.377 INFO:journalctl@ceph.mds.h.smithi089.stdout:Sep 21 01:35:26 smithi089 bash[68380]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-h 2024-09-21T01:35:26.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T01:35:26.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:26.555 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T01:35:26.556 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T01:35:26.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.k 2024-09-21T01:35:27.000 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:35:26 smithi089 systemd[1]: Stopping Ceph mds.k for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:27.000 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:35:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k[62578]: 2024-09-21T01:35:26.863+0000 7fd415025640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:27.000 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:35:26 smithi089 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k[62578]: 2024-09-21T01:35:26.863+0000 7fd415025640 -1 mds.k *** got signal Terminated *** 2024-09-21T01:35:30.000 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:35:29 smithi089 podman[68502]: 2024-09-21 01:35:29.728577347 +0000 UTC m=+2.956174054 container died 8e78bce89c243b91d89e67e372106ddfd00fd80a989acc805547080d1e0d83c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T01:35:30.472 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:35:30 smithi089 podman[68502]: 2024-09-21 01:35:30.221294712 +0000 UTC m=+3.448891419 container cleanup 8e78bce89c243b91d89e67e372106ddfd00fd80a989acc805547080d1e0d83c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T01:35:30.472 INFO:journalctl@ceph.mds.k.smithi089.stdout:Sep 21 01:35:30 smithi089 bash[68502]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-k 2024-09-21T01:35:30.603 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T01:35:30.633 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:30.634 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T01:35:30.634 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T01:35:30.634 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.c 2024-09-21T01:35:30.957 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:35:30 smithi098 systemd[1]: Stopping Ceph mds.c for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:31.211 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:35:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c[63802]: 2024-09-21T01:35:30.956+0000 7f741e7c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:31.211 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:35:30 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c[63802]: 2024-09-21T01:35:30.956+0000 7f741e7c8640 -1 mds.c *** got signal Terminated *** 2024-09-21T01:35:34.049 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:35:33 smithi098 podman[68581]: 2024-09-21 01:35:33.74236876 +0000 UTC m=+2.910060167 container died 124e199b88089c8c9cc4af22e3a54a09c784e5d56ef3ac120dbb026e1879fa1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T01:35:34.366 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:35:34 smithi098 podman[68581]: 2024-09-21 01:35:34.148025326 +0000 UTC m=+3.315716732 container cleanup 124e199b88089c8c9cc4af22e3a54a09c784e5d56ef3ac120dbb026e1879fa1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T01:35:34.366 INFO:journalctl@ceph.mds.c.smithi098.stdout:Sep 21 01:35:34 smithi098 bash[68581]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-c 2024-09-21T01:35:34.580 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T01:35:34.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:34.616 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T01:35:34.616 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T01:35:34.616 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.f 2024-09-21T01:35:34.944 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:35:34 smithi098 systemd[1]: Stopping Ceph mds.f for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:35.397 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:35:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f[63538]: 2024-09-21T01:35:34.944+0000 7f80969eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:35.398 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:35:34 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f[63538]: 2024-09-21T01:35:34.944+0000 7f80969eb640 -1 mds.f *** got signal Terminated *** 2024-09-21T01:35:38.003 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:35:37 smithi098 podman[68712]: 2024-09-21 01:35:37.739302568 +0000 UTC m=+2.904116028 container died cd07f533729873916d67a1cddb058ee23838a9fe077a37dbbeec92fc3a3a7200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T01:35:38.361 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:35:38 smithi098 podman[68712]: 2024-09-21 01:35:38.102524291 +0000 UTC m=+3.267337751 container cleanup cd07f533729873916d67a1cddb058ee23838a9fe077a37dbbeec92fc3a3a7200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T01:35:38.361 INFO:journalctl@ceph.mds.f.smithi098.stdout:Sep 21 01:35:38 smithi098 bash[68712]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-f 2024-09-21T01:35:38.521 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T01:35:38.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:38.555 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T01:35:38.555 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T01:35:38.555 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.i 2024-09-21T01:35:38.873 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:35:38 smithi098 systemd[1]: Stopping Ceph mds.i for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:39.147 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:35:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i[64075]: 2024-09-21T01:35:38.872+0000 7fb153b78640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:39.147 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:35:38 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i[64075]: 2024-09-21T01:35:38.872+0000 7fb153b78640 -1 mds.i *** got signal Terminated *** 2024-09-21T01:35:42.015 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:35:41 smithi098 podman[68836]: 2024-09-21 01:35:41.733644327 +0000 UTC m=+2.960200725 container died 5edfd74fe830e5c65f634d8b906622e5cf9f4ac26ef8a71beb5afce946924c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T01:35:42.357 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:35:42 smithi098 podman[68836]: 2024-09-21 01:35:42.097819169 +0000 UTC m=+3.324375547 container cleanup 5edfd74fe830e5c65f634d8b906622e5cf9f4ac26ef8a71beb5afce946924c37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T01:35:42.357 INFO:journalctl@ceph.mds.i.smithi098.stdout:Sep 21 01:35:42 smithi098 bash[68836]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-i 2024-09-21T01:35:42.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T01:35:42.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:42.561 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T01:35:42.561 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T01:35:42.561 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.l 2024-09-21T01:35:42.897 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:35:42 smithi098 systemd[1]: Stopping Ceph mds.l for fe42ec90-77b5-11ef-baf5-efdc52797490... 2024-09-21T01:35:42.898 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:35:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l[63266]: 2024-09-21T01:35:42.868+0000 7f4ea7d11640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T01:35:42.898 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:35:42 smithi098 ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l[63266]: 2024-09-21T01:35:42.868+0000 7f4ea7d11640 -1 mds.l *** got signal Terminated *** 2024-09-21T01:35:47.144 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:35:46 smithi098 podman[68956]: 2024-09-21 01:35:46.883595691 +0000 UTC m=+4.110166948 container died 72874aa4d923458bad0bb6be5bd948c8262010e3d4b77044f61d7f6f7c1bf80f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T01:35:47.397 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:35:47 smithi098 podman[68956]: 2024-09-21 01:35:47.244453077 +0000 UTC m=+4.471024337 container cleanup 72874aa4d923458bad0bb6be5bd948c8262010e3d4b77044f61d7f6f7c1bf80f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T01:35:47.398 INFO:journalctl@ceph.mds.l.smithi098.stdout:Sep 21 01:35:47 smithi098 bash[68956]: ceph-fe42ec90-77b5-11ef-baf5-efdc52797490-mds-l 2024-09-21T01:35:47.656 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fe42ec90-77b5-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T01:35:47.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T01:35:47.688 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T01:35:47.688 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T01:35:47.976 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:35:52.058 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T01:35:52.260 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:35:54.775 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T01:35:54.977 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:35:57.359 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:35:57.426 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:35:57.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T01:35:57.484 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T01:35:57.486 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi027/crash 2024-09-21T01:35:57.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash -- . 2024-09-21T01:35:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T01:35:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-09-21T01:35:57.561 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi089/crash 2024-09-21T01:35:57.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash -- . 2024-09-21T01:35:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T01:35:57.602 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2024-09-21T01:35:57.604 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi098/crash 2024-09-21T01:35:57.605 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash -- . 2024-09-21T01:35:57.644 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T01:35:57.645 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2024-09-21T01:35:57.646 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T01:35:57.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T01:35:57.730 INFO:tasks.cephadm:Compressing logs... 2024-09-21T01:35:57.731 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T01:35:57.734 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T01:35:57.737 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T01:35:57.762 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T01:35:57.762 INFO:teuthology.orchestra.run.smithi098.stderr:: No such file or directory 2024-09-21T01:35:57.762 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T01:35:57.763 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T01:35:57.763 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T01:35:57.763 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T01:35:57.763 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T01:35:57.763 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T01:35:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T01:35:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:: No such file or directory 2024-09-21T01:35:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T01:35:57.766 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T01:35:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T01:35:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T01:35:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T01:35:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T01:35:57.767 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T01:35:57.768 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.11.log: 81.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log: 90.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.l.log: 87.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T01:35:57.769 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T01:35:57.773 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.8.log: 95.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T01:35:57.773 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T01:35:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log: 90.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T01:35:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T01:35:57.773 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log: 81.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T01:35:57.774 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log 2024-09-21T01:35:57.774 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.z.log: 90.8% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T01:35:57.779 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T01:35:57.780 INFO:teuthology.orchestra.run.smithi089.stderr: 92.5% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T01:35:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T01:35:57.780 INFO:teuthology.orchestra.run.smithi098.stderr: 92.7% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T01:35:57.780 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T01:35:57.781 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T01:35:57.782 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T01:35:57.783 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.h.log: 87.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T01:35:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-21T01:35:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:: No such file or directory 2024-09-21T01:35:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T01:35:57.787 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T01:35:57.788 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T01:35:57.789 INFO:teuthology.orchestra.run.smithi027.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T01:35:57.789 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T01:35:57.790 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T01:35:57.792 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T01:35:57.793 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T01:35:57.793 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T01:35:57.794 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T01:35:57.795 INFO:teuthology.orchestra.run.smithi027.stderr: 84.1% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T01:35:57.795 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.a.log: 90.0%gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T01:35:57.796 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T01:35:57.796 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T01:35:57.798 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.g.log: /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log 2024-09-21T01:35:57.800 INFO:teuthology.orchestra.run.smithi027.stderr: 95.0%/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log: -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T01:35:57.801 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T01:35:57.803 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T01:35:57.803 INFO:teuthology.orchestra.run.smithi027.stderr: 87.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T01:35:57.805 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T01:35:57.806 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T01:35:57.806 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T01:35:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.j.log: -5 --verbose -- /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T01:35:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T01:35:57.828 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T01:35:57.836 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T01:35:57.837 INFO:teuthology.orchestra.run.smithi098.stderr: 93.3% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T01:35:57.852 INFO:teuthology.orchestra.run.smithi089.stderr: 94.3% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T01:35:57.859 INFO:teuthology.orchestra.run.smithi098.stderr: 93.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T01:35:57.866 INFO:teuthology.orchestra.run.smithi098.stderr: 94.4% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T01:35:57.880 INFO:teuthology.orchestra.run.smithi027.stderr: 93.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T01:35:57.889 INFO:teuthology.orchestra.run.smithi089.stderr: 93.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T01:35:57.905 INFO:teuthology.orchestra.run.smithi027.stderr: 94.3% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T01:35:57.908 INFO:teuthology.orchestra.run.smithi027.stderr: 94.3% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T01:35:57.922 INFO:teuthology.orchestra.run.smithi089.stderr: 93.6% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T01:35:58.131 INFO:teuthology.orchestra.run.smithi098.stderr: 91.7% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T01:35:58.247 INFO:teuthology.orchestra.run.smithi027.stderr: 90.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T01:35:58.337 INFO:teuthology.orchestra.run.smithi089.stderr: 91.8% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T01:35:58.347 INFO:teuthology.orchestra.run.smithi089.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T01:35:59.182 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T01:36:03.156 INFO:teuthology.orchestra.run.smithi089.stderr: 93.1% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T01:36:03.869 INFO:teuthology.orchestra.run.smithi098.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T01:36:03.977 INFO:teuthology.orchestra.run.smithi098.stderr: 93.2% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T01:36:04.757 INFO:teuthology.orchestra.run.smithi089.stderr: 93.1% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T01:36:05.377 INFO:teuthology.orchestra.run.smithi098.stderr: 93.1% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T01:36:05.975 INFO:teuthology.orchestra.run.smithi027.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T01:36:06.612 INFO:teuthology.orchestra.run.smithi089.stderr: 93.1% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T01:36:07.075 INFO:teuthology.orchestra.run.smithi027.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T01:36:07.367 INFO:teuthology.orchestra.run.smithi089.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T01:36:07.368 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-09-21T01:36:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m9.617s 2024-09-21T01:36:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m31.020s 2024-09-21T01:36:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m1.088s 2024-09-21T01:36:07.523 INFO:teuthology.orchestra.run.smithi027.stderr: 93.1% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T01:36:07.971 INFO:teuthology.orchestra.run.smithi098.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T01:36:07.972 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-21T01:36:07.972 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m10.221s 2024-09-21T01:36:07.972 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m29.706s 2024-09-21T01:36:07.972 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m1.045s 2024-09-21T01:36:09.439 INFO:teuthology.orchestra.run.smithi027.stderr: 93.0% -- replaced with /var/log/ceph/fe42ec90-77b5-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T01:36:09.442 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-09-21T01:36:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m11.678s 2024-09-21T01:36:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m37.439s 2024-09-21T01:36:09.442 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m3.211s 2024-09-21T01:36:09.443 INFO:tasks.cephadm:Archiving logs... 2024-09-21T01:36:09.443 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi027/log 2024-09-21T01:36:09.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T01:36:15.075 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi089/log 2024-09-21T01:36:15.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T01:36:19.722 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi098/log 2024-09-21T01:36:19.722 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T01:36:24.320 INFO:tasks.cephadm:Removing cluster... 2024-09-21T01:36:24.320 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --force 2024-09-21T01:36:24.613 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:36:25.261 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --force 2024-09-21T01:36:25.468 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:36:25.791 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fe42ec90-77b5-11ef-baf5-efdc52797490 --force 2024-09-21T01:36:25.995 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: fe42ec90-77b5-11ef-baf5-efdc52797490 2024-09-21T01:36:26.299 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T01:36:26.299 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:26.335 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:26.352 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T01:36:26.368 INFO:tasks.cephadm:Teardown complete 2024-09-21T01:36:26.368 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T01:36:26.378 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T01:36:26.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T01:36:26.382 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T01:36:26.395 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T01:36:26.507 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T01:36:26.518 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T01:36:26.518 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:36:26.522 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:36:26.525 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T01:36:26.536 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:36:26.538 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:36:26.540 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:36:26.541 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-09-21T01:36:26.541 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +313us[ +318us] +/- 36ms 2024-09-21T01:36:26.541 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 -1158us[-1153us] +/- 79ms 2024-09-21T01:36:26.541 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +535us[ +540us] +/- 84ms 2024-09-21T01:36:26.541 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:36:26.543 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:36:26.543 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-09-21T01:36:26.543 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 +392us[ +390us] +/- 36ms 2024-09-21T01:36:26.543 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 48 -1054us[-1056us] +/- 79ms 2024-09-21T01:36:26.543 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +612us[ +611us] +/- 84ms 2024-09-21T01:36:26.543 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:36:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-09-21T01:36:26.560 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T01:36:26.560 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-09-21T01:36:26.560 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +323us[ +344us] +/- 36ms 2024-09-21T01:36:26.560 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1173us[-1152us] +/- 79ms 2024-09-21T01:36:26.560 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +561us[ +581us] +/- 84ms 2024-09-21T01:36:26.560 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T01:36:26.561 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T01:36:26.571 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T01:36:26.571 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T01:36:26.584 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T01:36:26.734 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T01:36:26.800 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T01:36:26.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:36:26.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:36:26.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:36:26.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:36:26.955 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T01:36:26.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:36:26.983 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T01:36:26.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T01:36:27.003 INFO:teuthology.task.internal:Duration was 2084.954957 seconds 2024-09-21T01:36:27.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T01:36:27.013 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T01:36:27.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T01:36:27.018 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T01:36:27.020 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T01:36:27.057 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:36:27.061 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:36:27.103 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T01:36:27.496 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T01:36:27.496 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-09-21T01:36:27.497 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T01:36:27.549 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-09-21T01:36:27.549 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T01:36:27.588 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-09-21T01:36:27.588 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T01:36:27.625 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T01:36:27.625 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T01:36:27.630 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T01:36:27.632 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T01:36:27.692 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T01:36:27.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T01:36:27.696 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T01:36:27.699 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T01:36:28.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T01:36:28.474 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T01:36:28.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T01:36:28.532 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T01:36:28.559 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T01:36:28.586 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T01:36:28.597 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T01:36:28.601 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T01:36:28.604 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T01:36:28.627 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-09-21T01:36:28.651 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-09-21T01:36:28.669 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-09-21T01:36:28.700 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T01:36:28.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:36:28.735 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T01:36:28.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:36:28.751 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T01:36:28.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T01:36:28.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T01:36:28.777 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T01:36:28.777 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi027 2024-09-21T01:36:28.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T01:36:28.859 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi089 2024-09-21T01:36:28.859 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T01:36:28.897 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913359/remote/smithi098 2024-09-21T01:36:28.897 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T01:36:28.933 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T01:36:28.934 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T01:36:28.938 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T01:36:28.940 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T01:36:28.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T01:36:28.997 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T01:36:28.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T01:36:29.008 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T01:36:29.008 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T01:36:29.015 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T01:36:29.017 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T01:36:29.031 INFO:teuthology.orchestra.run.smithi089.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 01:36 /home/ubuntu/cephtest 2024-09-21T01:36:29.039 INFO:teuthology.orchestra.run.smithi098.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 01:36 /home/ubuntu/cephtest 2024-09-21T01:36:29.047 INFO:teuthology.orchestra.run.smithi027.stdout: 393956 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 01:36 /home/ubuntu/cephtest 2024-09-21T01:36:29.049 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T01:36:29.060 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T01:36:29.086 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 2084.954957485199 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T01:36:29.086 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T01:36:29.166 INFO:teuthology.run:pass